2023-09-29T18:08:42.117 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-29T18:08:42.118 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834 --owner scheduled_yuriw@teuthology --description rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} --name yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834/orig.config.yaml 2023-09-29T18:08:42.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:08:42.250 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834 branch: wip-yuri3-testing-2023-09-27-0919-quincy description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} email: null first_in_suite: false job_id: '7405834' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-09-27-0919-quincy ceph: conf: client: rbd default format: 1 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-09-27-0919-quincy sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1925 sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuri3-testing-2023-09-27-0919-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1dF7bKZeldZ/9Dl+g0zDsW2d5rAyu7pDHG6+AZnv2ETsbs1GwvhTB0ctyT6q5rNSs11+G17pq8WTLQX3HB918= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - workunit: clients: client.0: - rbd/cli_generic.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-28_23:00:03 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-29T18:08:42.251 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa; will attempt to use it 2023-09-29T18:08:42.251 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks 2023-09-29T18:08:42.252 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-29T18:08:42.253 INFO:teuthology.task.internal:Checking packages... 2023-09-29T18:08:42.277 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2c186684db7f9ea0cdb098fc86b350a0fdd26eba' 2023-09-29T18:08:42.277 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-29T18:08:42.277 INFO:teuthology.packaging:ref: None 2023-09-29T18:08:42.278 INFO:teuthology.packaging:tag: None 2023-09-29T18:08:42.278 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-09-27-0919-quincy 2023-09-29T18:08:42.278 INFO:teuthology.packaging:sha1: 2c186684db7f9ea0cdb098fc86b350a0fdd26eba 2023-09-29T18:08:42.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-09-27-0919-quincy 2023-09-29T18:08:42.416 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1143.g2c186684 2023-09-29T18:08:42.416 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-29T18:08:42.422 INFO:teuthology.task.internal:no buildpackages task found 2023-09-29T18:08:42.422 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-29T18:08:42.443 INFO:teuthology.task.internal:Saving configuration 2023-09-29T18:08:42.455 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-29T18:08:42.461 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T18:08:42.486 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 18:03:50.939568', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1dF7bKZeldZ/9Dl+g0zDsW2d5rAyu7pDHG6+AZnv2ETsbs1GwvhTB0ctyT6q5rNSs11+G17pq8WTLQX3HB918='} 2023-09-29T18:08:42.487 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-29T18:08:42.493 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-09-29T18:08:42.493 INFO:teuthology.run_tasks:Running task console_log... 2023-09-29T18:08:42.536 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f34cf220ee0>, signals=[15]) 2023-09-29T18:08:42.537 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-29T18:08:42.542 INFO:teuthology.task.internal:Opening connections... 2023-09-29T18:08:42.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:08:42.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:08:42.611 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-29T18:08:42.617 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2023-09-29T18:08:42.677 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2023-09-29T18:08:42.678 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2023-09-29T18:08:42.731 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.4" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T18:08:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T18:08:42.733 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T18:08:42.734 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2023-09-29T18:08:42.756 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-29T18:08:42.764 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-29T18:08:42.770 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-29T18:08:42.770 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2023-09-29T18:08:42.784 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-29T18:08:42.790 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-29T18:08:42.790 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2023-09-29T18:08:42.839 INFO:teuthology.orchestra.run.smithi143.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-09-29T18:08:42.839 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-29T18:08:42.871 INFO:teuthology.run_tasks:Running task kernel... 2023-09-29T18:08:42.885 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-29T18:08:42.885 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-29T18:08:42.886 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-29T18:08:42.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T18:08:42.899 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-09-29T18:08:42.954 INFO:teuthology.orchestra.run.smithi143.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-09-29T18:08:42.954 INFO:teuthology.task.kernel:Running kernel on smithi143: 4.18.0-305.25.1.el8_4.x86_64 2023-09-29T18:08:42.954 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum install -y kernel 2023-09-29T18:08:44.478 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:08:44.479 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-09-29T18:08:46.052 INFO:teuthology.orchestra.run.smithi143.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-09-29T18:08:46.552 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 32 MB/s | 16 MB 00:00 2023-09-29T18:08:49.847 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-09-29T18:08:50.596 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-09-29T18:08:50.596 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-09-29T18:08:50.596 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-09-29T18:08:50.753 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:08:50.754 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:08:50.754 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:08:50.869 DEBUG:teuthology.orchestra.run.smithi143:> echo no | sudo yum reinstall kernel || true 2023-09-29T18:08:51.212 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:08:51.213 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-09-29T18:08:51.506 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:02 ago on Fri 29 Sep 2023 06:08:49 PM UTC. 2023-09-29T18:08:51.640 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-09-29T18:08:51.640 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-09-29T18:08:51.640 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-09-29T18:08:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:Error: No packages marked for reinstall. 2023-09-29T18:08:51.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum reinstall -y kernel || true 2023-09-29T18:08:52.064 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:08:52.064 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-09-29T18:08:52.345 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:03 ago on Fri 29 Sep 2023 06:08:49 PM UTC. 2023-09-29T18:08:52.496 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-09-29T18:08:52.496 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-09-29T18:08:52.496 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-09-29T18:08:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:Error: No packages marked for reinstall. 2023-09-29T18:08:52.568 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2023-09-29T18:08:52.693 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.694 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.694 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-29T18:08:52.694 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-29T18:08:52.694 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-29T18:08:52.694 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2023-09-29T18:08:52.695 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2023-09-29T18:08:52.819 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.819 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2023-09-29T18:08:52.846 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2023-09-29T18:08:52.846 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2023-09-29T18:08:52.846 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.02 2023-09-29T18:08:52.846 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 99.el8_4.1 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Group : System Environment/Base 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 9264631 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Relocations : (not relocatable) 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : Red Hat, Inc. 2023-09-29T18:08:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : Red Hat, Inc. 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:08:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2023-09-29T18:08:52.850 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-29T18:08:52.850 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:52.850 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-29T18:08:54.281 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2023-09-29T18:08:56.828 INFO:teuthology.orchestra.run.smithi143.stderr:done 2023-09-29T18:08:56.828 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-09-29T18:08:56.842 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.9ZPFMxeNJr 2023-09-29T18:08:56.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9ZPFMxeNJr 2023-09-29T18:08:56.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.9ZPFMxeNJr 2023-09-29T18:08:57.066 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.9ZPFMxeNJr is 6KB 2023-09-29T18:08:57.115 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.9ZPFMxeNJr 2023-09-29T18:08:57.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2023-09-29T18:08:57.199 INFO:teuthology.orchestra.run.smithi143.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-09-29T18:08:57.199 INFO:teuthology.orchestra.run.smithi143.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-09-29T18:08:57.199 INFO:teuthology.orchestra.run.smithi143.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-09-29T18:08:57.199 INFO:teuthology.orchestra.run.smithi143.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-09-29T18:08:57.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-09-29T18:08:57.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2023-09-29T18:09:27.382 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-29T18:09:27.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:09:27.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:09:45.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T18:09:54.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:09:54.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:09:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T18:10:09.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:10:09.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:10:28.346 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T18:10:43.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:10:43.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:10:43.864 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-09-29T18:10:44.120 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:10:44.120 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-425.3.1.el8.x86_64"... 2023-09-29T18:10:44.120 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-09-29T18:10:44.135 INFO:teuthology.orchestra.run.smithi143.stdout:4.18.0-425.3.1.el8.x86_64 2023-09-29T18:10:44.135 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-09-29T18:10:44.135 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-29T18:10:44.135 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-09-29T18:10:45.137 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-29T18:10:45.137 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-29T18:10:45.909 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2023-09-29T18:10:45.951 DEBUG:teuthology.parallel:result is None 2023-09-29T18:10:45.951 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-29T18:10:45.963 INFO:teuthology.task.internal:Creating test directory... 2023-09-29T18:10:45.964 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-29T18:10:46.341 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-29T18:10:46.348 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-29T18:10:46.356 INFO:teuthology.task.internal:Creating archive directory... 2023-09-29T18:10:46.356 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-29T18:10:46.485 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-29T18:10:46.493 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-29T18:10:46.494 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-29T18:10:46.981 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T18:10:47.254 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T18:10:47.274 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-29T18:10:47.283 INFO:teuthology.task.internal:Configuring sudo... 2023-09-29T18:10:47.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-29T18:10:47.341 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-29T18:10:47.353 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-29T18:10:47.354 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-29T18:10:47.369 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-29T18:10:47.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-29T18:10:47.632 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-29T18:10:47.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-29T18:10:47.714 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:10:47.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-29T18:10:47.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2023-09-29T18:10:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-29T18:10:48.135 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-29T18:10:48.144 INFO:teuthology.task.internal:Starting timer... 2023-09-29T18:10:48.144 INFO:teuthology.run_tasks:Running task pcp... 2023-09-29T18:10:48.157 INFO:teuthology.run_tasks:Running task selinux... 2023-09-29T18:10:48.166 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-29T18:10:48.166 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2023-09-29T18:10:48.301 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: [ OK ] 2023-09-29T18:10:48.303 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-29T18:10:48.303 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2023-09-29T18:10:48.317 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2023-09-29T18:10:48.317 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2023-09-29T18:10:48.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-29T18:10:48.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T18:10:48.383 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-29T18:10:48.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2023-09-29T18:10:48.447 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-29T18:10:48.457 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-29T18:10:48.463 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-29T18:10:48.615 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-29T18:10:48.631 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-29T18:10:48.632 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-29T18:19:55.635 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2023-09-29T18:19:55.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:19:55.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T18:19:55.706 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-09-29T18:19:55.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T18:19:55.778 INFO:teuthology.run_tasks:Running task clock... 2023-09-29T18:19:55.788 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-29T18:19:55.789 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-29T18:19:55.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T18:19:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-29T18:19:55.861 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-29T18:19:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2023-09-29T18:19:55.915 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2023-09-29T18:19:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-29T18:19:55.942 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-29T18:19:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:bash: ntpq: command not found 2023-09-29T18:19:56.009 INFO:teuthology.orchestra.run.smithi143.stdout:210 Number of sources = 4 2023-09-29T18:19:56.009 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-29T18:19:56.010 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2023-09-29T18:19:56.010 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-29T18:19:56.010 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-29T18:19:56.010 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-29T18:19:56.010 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-29T18:19:56.010 INFO:teuthology.run_tasks:Running task install... 2023-09-29T18:19:56.021 DEBUG:teuthology.task.install:project ceph 2023-09-29T18:19:56.021 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'}} 2023-09-29T18:19:56.021 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'} 2023-09-29T18:19:56.022 INFO:teuthology.task.install:Using flavor: default 2023-09-29T18:19:56.027 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-29T18:19:56.027 INFO:teuthology.task.install:extra packages: [] 2023-09-29T18:19:56.028 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba', 'tag': None, 'wait_for_package': False} 2023-09-29T18:19:56.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2c186684db7f9ea0cdb098fc86b350a0fdd26eba 2023-09-29T18:19:56.171 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-09-27-0919-quincy/2c186684db7f9ea0cdb098fc86b350a0fdd26eba/centos/8/flavors/default/ 2023-09-29T18:19:56.172 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1143.g2c186684 2023-09-29T18:19:56.347 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-09-27-0919-quincy/2c186684db7f9ea0cdb098fc86b350a0fdd26eba/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-09-27-0919-quincy/2c186684db7f9ea0cdb098fc86b350a0fdd26eba/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-09-27-0919-quincy/2c186684db7f9ea0cdb098fc86b350a0fdd26eba/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-29T18:19:56.347 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:19:56.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-29T18:19:56.372 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-09-29T18:19:56.372 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2c186684db7f9ea0cdb098fc86b350a0fdd26eba/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-29T18:19:56.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-29T18:19:56.512 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-29T18:19:56.576 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2023-09-29T18:19:56.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2023-09-29T18:19:56.965 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:19:57.438 INFO:teuthology.orchestra.run.smithi143.stdout:44 files removed 2023-09-29T18:19:57.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw 2023-09-29T18:19:57.844 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:19:58.804 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 202 kB/s | 78 kB 00:00 2023-09-29T18:19:59.092 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-09-29T18:19:59.442 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-09-29T18:19:59.557 INFO:teuthology.orchestra.run.smithi143.stdout:Copr repo for python3-asyncssh owned by ceph 40 kB/s | 3.5 kB 00:00 2023-09-29T18:20:00.681 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-09-29T18:20:03.975 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-09-29T18:20:04.400 INFO:teuthology.orchestra.run.smithi143.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 MB/s | 33 MB 00:00 2023-09-29T18:20:09.517 INFO:teuthology.orchestra.run.smithi143.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 119 MB/s | 38 MB 00:00 2023-09-29T18:20:19.037 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout:==================================================================================================== 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout:==================================================================================================== 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:17.2.6-1143.g2c186684.el8 ceph 17 M 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2023-09-29T18:20:19.040 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:17.2.6-1143.g2c186684.el8 ceph 3.5 M 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:17.2.6-1143.g2c186684.el8 ceph 3.6 M 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:17.2.6-1143.g2c186684.el8 ceph 5.2 M 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:17.2.6-1143.g2c186684.el8 ceph 23 M 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:17.2.6-1143.g2c186684.el8 ceph 24 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:17.2.6-1143.g2c186684.el8 ceph 796 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:17.2.6-1143.g2c186684.el8 ceph 506 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-09-29T18:20:19.041 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:17.2.6-1143.g2c186684.el8 ceph 5.5 M 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:17.2.6-1143.g2c186684.el8 ceph 45 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:17.2.6-1143.g2c186684.el8 ceph 115 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:17.2.6-1143.g2c186684.el8 ceph 191 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:17.2.6-1143.g2c186684.el8 ceph 370 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:17.2.6-1143.g2c186684.el8 ceph 358 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:17.2.6-1143.g2c186684.el8 ceph 106 k 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-29T18:20:19.042 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:==================================================================================================== 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:Install 21 Packages 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 2 Packages 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:20:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 63 M 2023-09-29T18:20:19.044 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:20:19.526 INFO:teuthology.orchestra.run.smithi143.stdout:(1/23): ceph-base-17.2.6-1143.g2c186684.el8.x86 12 MB/s | 5.2 MB 00:00 2023-09-29T18:20:19.551 INFO:teuthology.orchestra.run.smithi143.stdout:(2/23): ceph-selinux-17.2.6-1143.g2c186684.el8. 993 kB/s | 24 kB 00:00 2023-09-29T18:20:19.593 INFO:teuthology.orchestra.run.smithi143.stdout:(3/23): libcephfs2-17.2.6-1143.g2c186684.el8.x8 19 MB/s | 796 kB 00:00 2023-09-29T18:20:19.624 INFO:teuthology.orchestra.run.smithi143.stdout:(4/23): libradosstriper1-17.2.6-1143.g2c186684. 16 MB/s | 506 kB 00:00 2023-09-29T18:20:19.887 INFO:teuthology.orchestra.run.smithi143.stdout:(5/23): librgw2-17.2.6-1143.g2c186684.el8.x86_6 21 MB/s | 5.5 MB 00:00 2023-09-29T18:20:19.910 INFO:teuthology.orchestra.run.smithi143.stdout:(6/23): python3-ceph-argparse-17.2.6-1143.g2c18 1.9 MB/s | 45 kB 00:00 2023-09-29T18:20:19.935 INFO:teuthology.orchestra.run.smithi143.stdout:(7/23): python3-ceph-common-17.2.6-1143.g2c1866 4.7 MB/s | 115 kB 00:00 2023-09-29T18:20:19.959 INFO:teuthology.orchestra.run.smithi143.stdout:(8/23): python3-cephfs-17.2.6-1143.g2c186684.el 7.7 MB/s | 191 kB 00:00 2023-09-29T18:20:20.002 INFO:teuthology.orchestra.run.smithi143.stdout:(9/23): python3-rados-17.2.6-1143.g2c186684.el8 8.6 MB/s | 370 kB 00:00 2023-09-29T18:20:20.032 INFO:teuthology.orchestra.run.smithi143.stdout:(10/23): python3-rbd-17.2.6-1143.g2c186684.el8. 12 MB/s | 358 kB 00:00 2023-09-29T18:20:20.080 INFO:teuthology.orchestra.run.smithi143.stdout:(11/23): python3-rgw-17.2.6-1143.g2c186684.el8. 2.2 MB/s | 106 kB 00:00 2023-09-29T18:20:20.569 INFO:teuthology.orchestra.run.smithi143.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 626 kB/s | 306 kB 00:00 2023-09-29T18:20:20.651 INFO:teuthology.orchestra.run.smithi143.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 726 kB/s | 59 kB 00:00 2023-09-29T18:20:20.735 INFO:teuthology.orchestra.run.smithi143.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 895 kB/s | 75 kB 00:00 2023-09-29T18:20:21.010 INFO:teuthology.orchestra.run.smithi143.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-09-29T18:20:21.038 INFO:teuthology.orchestra.run.smithi143.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 1.6 MB/s | 44 kB 00:00 2023-09-29T18:20:21.052 INFO:teuthology.orchestra.run.smithi143.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 18 MB/s | 259 kB 00:00 2023-09-29T18:20:21.068 INFO:teuthology.orchestra.run.smithi143.stdout:(18/23): librdkafka-0.11.4-1.el8.x86_64.rpm 22 MB/s | 353 kB 00:00 2023-09-29T18:20:21.080 INFO:teuthology.orchestra.run.smithi143.stdout:(19/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.7 MB/s | 47 kB 00:00 2023-09-29T18:20:21.611 INFO:teuthology.orchestra.run.smithi143.stdout:(20/23): librados2-17.2.6-1143.g2c186684.el8.x8 6.5 MB/s | 3.5 MB 00:00 2023-09-29T18:20:21.784 INFO:teuthology.orchestra.run.smithi143.stdout:(21/23): ceph-common-17.2.6-1143.g2c186684.el8. 8.7 MB/s | 23 MB 00:02 2023-09-29T18:20:21.924 INFO:teuthology.orchestra.run.smithi143.stdout:(22/23): librbd1-17.2.6-1143.g2c186684.el8.x86_ 11 MB/s | 3.6 MB 00:00 2023-09-29T18:20:22.148 INFO:teuthology.orchestra.run.smithi143.stdout:(23/23): ceph-radosgw-17.2.6-1143.g2c186684.el8 5.5 MB/s | 17 MB 00:03 2023-09-29T18:20:22.151 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:20:22.151 INFO:teuthology.orchestra.run.smithi143.stdout:Total 20 MB/s | 63 MB 00:03 2023-09-29T18:20:22.155 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:20:22.200 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:20:22.200 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:20:22.949 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:20:22.954 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:20:24.060 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:20:24.253 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-09-29T18:20:24.621 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-09-29T18:20:24.836 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:17.2.6-1143.g2c186684.el8.x86_64 2/25 2023-09-29T18:20:24.938 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:17.2.6-1143.g2c186684.el8.x86_64 2/25 2023-09-29T18:20:25.117 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:17.2.6-1143.g2c186684.el8.x86_64 3/25 2023-09-29T18:20:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-09-29T18:20:25.553 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:17.2.6-1143.g2c186684.el8.x86_64 5/25 2023-09-29T18:20:25.957 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:17.2.6-1143.g2c186684.el8.x86_64 5/25 2023-09-29T18:20:26.161 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:17.2.6-1143.g2c186684.el8.x86_64 6/25 2023-09-29T18:20:26.228 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:17.2.6-1143.g2c186684.el8.x86_64 6/25 2023-09-29T18:20:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-09-29T18:20:26.603 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-09-29T18:20:26.805 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-09-29T18:20:27.456 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-09-29T18:20:27.611 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:17.2.6-1143.g2c186684.el8.x86_64 10/25 2023-09-29T18:20:27.684 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:17.2.6-1143.g2c186684.el8.x86_64 10/25 2023-09-29T18:20:27.805 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:17.2.6-1143.g2c186684.el8. 11/25 2023-09-29T18:20:27.981 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:17.2.6-1143.g2c186684.el8.x86_64 12/25 2023-09-29T18:20:28.196 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:17.2.6-1143.g2c186684.el8.x86_64 13/25 2023-09-29T18:20:28.415 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:17.2.6-1143.g2c186684.el8.x86_64 14/25 2023-09-29T18:20:28.561 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:17.2.6-1143.g2c186684.el8.x86_6 15/25 2023-09-29T18:20:28.631 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1143.g2c186684.el8.x86_6 15/25 2023-09-29T18:20:28.788 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-09-29T18:20:28.955 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-09-29T18:20:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-09-29T18:20:29.370 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:17.2.6-1143.g2c186684.el8.x8 19/25 2023-09-29T18:20:34.388 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 20/25 2023-09-29T18:20:34.627 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 20/25 2023-09-29T18:20:35.206 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 20/25 2023-09-29T18:20:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:17.2.6-1143.g2c186684.el8.x86_64 21/25 2023-09-29T18:20:35.593 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:17.2.6-1143.g2c186684.el8.x86_64 21/25 2023-09-29T18:20:35.743 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:17.2.6-1143.g2c186684.el8.x86_64 22/25 2023-09-29T18:20:53.422 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1143.g2c186684.el8.x86_64 22/25 2023-09-29T18:20:53.422 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2023-09-29T18:20:53.422 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2023-09-29T18:20:53.422 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /dev 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /run 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /dev 2023-09-29T18:20:53.423 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:20:55.095 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:17.2.6-1143.g2c186684.el8.x86_64 23/25 2023-09-29T18:20:55.171 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1143.g2c186684.el8.x86_64 23/25 2023-09-29T18:20:55.172 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-29T18:20:55.375 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-09-29T18:20:55.376 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-29T18:20:56.094 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-09-29T18:20:56.094 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:17.2.6-1143.g2c186684.el8.x86_64 1/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 2/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:17.2.6-1143.g2c186684.el8.x86_64 3/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:17.2.6-1143.g2c186684.el8.x86_64 4/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:17.2.6-1143.g2c186684.el8.x86_64 5/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:17.2.6-1143.g2c186684.el8.x86_6 6/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:17.2.6-1143.g2c186684.el8.x86_64 7/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1143.g2c186684.el8. 8/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:17.2.6-1143.g2c186684.el8.x8 9/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:17.2.6-1143.g2c186684.el8.x86_64 10/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:17.2.6-1143.g2c186684.el8.x86_64 11/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:17.2.6-1143.g2c186684.el8.x86_64 12/25 2023-09-29T18:20:56.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:17.2.6-1143.g2c186684.el8.x86_64 13/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:17.2.6-1143.g2c186684.el8.x86_64 22/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-09-29T18:20:56.096 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:17.2.6-1143.g2c186684.el8.x86_64 24/25 2023-09-29T18:20:57.585 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-09-29T18:20:57.586 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:48 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:00.863 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.977 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.978 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:00.979 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:21:01.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-test 2023-09-29T18:21:01.483 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:21:03.325 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:54 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:05.124 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:17.2.6-1143.g2c186684.el8 ceph 52 M 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:21:05.125 INFO:teuthology.orchestra.run.smithi143.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:17.2.6-1143.g2c186684.el8 ceph 175 k 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout:Install 6 Packages 2023-09-29T18:21:05.126 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:05.127 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 53 M 2023-09-29T18:21:05.127 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 233 M 2023-09-29T18:21:05.127 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:21:05.131 INFO:teuthology.orchestra.run.smithi143.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 20 MB/s | 70 kB 00:00 2023-09-29T18:21:05.270 INFO:teuthology.orchestra.run.smithi143.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 1.1 MB/s | 161 kB 00:00 2023-09-29T18:21:05.282 INFO:teuthology.orchestra.run.smithi143.stdout:(3/6): libcephsqlite-17.2.6-1143.g2c186684.el8. 1.1 MB/s | 175 kB 00:00 2023-09-29T18:21:05.339 INFO:teuthology.orchestra.run.smithi143.stdout:(4/6): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-09-29T18:21:05.440 INFO:teuthology.orchestra.run.smithi143.stdout:(5/6): socat-1.7.3.3-2.el8.x86_64.rpm 1.9 MB/s | 302 kB 00:00 2023-09-29T18:21:07.800 INFO:teuthology.orchestra.run.smithi143.stdout:(6/6): ceph-test-17.2.6-1143.g2c186684.el8.x86_ 19 MB/s | 52 MB 00:02 2023-09-29T18:21:07.803 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:21:07.804 INFO:teuthology.orchestra.run.smithi143.stdout:Total 20 MB/s | 53 MB 00:02 2023-09-29T18:21:07.808 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:21:07.834 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:21:07.835 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:21:08.165 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:21:08.169 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:21:08.664 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:21:08.840 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-09-29T18:21:09.029 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-09-29T18:21:09.097 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-09-29T18:21:09.253 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-09-29T18:21:09.412 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-09-29T18:21:09.563 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:17.2.6-1143.g2c186684.el8.x86_64 5/6 2023-09-29T18:21:14.538 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1143.g2c186684.el8.x86_64 5/6 2023-09-29T18:21:14.834 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:17.2.6-1143.g2c186684.el8.x86_64 6/6 2023-09-29T18:21:15.101 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-test-2:17.2.6-1143.g2c186684.el8.x86_64 6/6 2023-09-29T18:21:15.102 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:17.2.6-1143.g2c186684.el8.x86_64 1/6 2023-09-29T18:21:15.102 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:17.2.6-1143.g2c186684.el8.x86_64 2/6 2023-09-29T18:21:15.102 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-09-29T18:21:15.102 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-09-29T18:21:15.102 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-09-29T18:21:16.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-09-29T18:21:16.260 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:07 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:18.462 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:21:18.559 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:18.559 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: jq-1.5-12.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.3.3-2.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:18.560 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:21:18.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph 2023-09-29T18:21:19.032 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:21:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:11 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:22.016 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:21:22.018 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================= 2023-09-29T18:21:22.018 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================= 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:17.2.6-1143.g2c186684.el8 ceph 6.4 k 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:17.2.6-1143.g2c186684.el8 ceph 2.3 M 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:17.2.6-1143.g2c186684.el8 ceph 1.6 M 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 238 k 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:17.2.6-1143.g2c186684.el8 ceph 3.9 M 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:17.2.6-1143.g2c186684.el8 ceph 17 M 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-29T18:21:22.019 INFO:teuthology.orchestra.run.smithi143.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-29T18:21:22.020 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:21:22.021 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================= 2023-09-29T18:21:22.022 INFO:teuthology.orchestra.run.smithi143.stdout:Install 28 Packages 2023-09-29T18:21:22.022 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:22.022 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 27 M 2023-09-29T18:21:22.022 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 106 M 2023-09-29T18:21:22.022 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:21:22.249 INFO:teuthology.orchestra.run.smithi143.stdout:(1/28): ceph-17.2.6-1143.g2c186684.el8.x86_64.r 39 kB/s | 6.4 kB 00:00 2023-09-29T18:21:22.494 INFO:teuthology.orchestra.run.smithi143.stdout:(2/28): ceph-mgr-17.2.6-1143.g2c186684.el8.x86_ 3.8 MB/s | 1.6 MB 00:00 2023-09-29T18:21:22.946 INFO:teuthology.orchestra.run.smithi143.stdout:(3/28): ceph-mds-17.2.6-1143.g2c186684.el8.x86_ 2.7 MB/s | 2.3 MB 00:00 2023-09-29T18:21:23.025 INFO:teuthology.orchestra.run.smithi143.stdout:(4/28): ceph-mgr-modules-core-17.2.6-1143.g2c18 3.0 MB/s | 238 kB 00:00 2023-09-29T18:21:23.237 INFO:teuthology.orchestra.run.smithi143.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 206 kB/s | 44 kB 00:00 2023-09-29T18:21:23.300 INFO:teuthology.orchestra.run.smithi143.stdout:(6/28): ceph-mon-17.2.6-1143.g2c186684.el8.x86_ 3.7 MB/s | 3.9 MB 00:01 2023-09-29T18:21:23.509 INFO:teuthology.orchestra.run.smithi143.stdout:(7/28): ceph-osd-17.2.6-1143.g2c186684.el8.x86_ 16 MB/s | 17 MB 00:01 2023-09-29T18:21:23.521 INFO:teuthology.orchestra.run.smithi143.stdout:(8/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 652 kB/s | 185 kB 00:00 2023-09-29T18:21:23.564 INFO:teuthology.orchestra.run.smithi143.stdout:(9/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 653 kB/s | 173 kB 00:00 2023-09-29T18:21:23.615 INFO:teuthology.orchestra.run.smithi143.stdout:(10/28): python3-jaraco-6.2-6.el8.noarch.rpm 119 kB/s | 11 kB 00:00 2023-09-29T18:21:23.677 INFO:teuthology.orchestra.run.smithi143.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 156 kB/s | 18 kB 00:00 2023-09-29T18:21:23.680 INFO:teuthology.orchestra.run.smithi143.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 748 kB/s | 49 kB 00:00 2023-09-29T18:21:23.730 INFO:teuthology.orchestra.run.smithi143.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-09-29T18:21:23.791 INFO:teuthology.orchestra.run.smithi143.stdout:(14/28): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-09-29T18:21:23.791 INFO:teuthology.orchestra.run.smithi143.stdout:(15/28): python3-portend-2.6-1.el8.noarch.rpm 254 kB/s | 16 kB 00:00 2023-09-29T18:21:23.821 INFO:teuthology.orchestra.run.smithi143.stdout:(16/28): python3-cherrypy-18.4.0-1.el8.noarch.r 1.2 MB/s | 384 kB 00:00 2023-09-29T18:21:23.854 INFO:teuthology.orchestra.run.smithi143.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 383 kB/s | 24 kB 00:00 2023-09-29T18:21:23.863 INFO:teuthology.orchestra.run.smithi143.stdout:(18/28): python3-simplegeneric-0.8.1-17.el8.noa 259 kB/s | 19 kB 00:00 2023-09-29T18:21:23.884 INFO:teuthology.orchestra.run.smithi143.stdout:(19/28): python3-tempora-1.14.1-5.el8.noarch.rp 469 kB/s | 29 kB 00:00 2023-09-29T18:21:23.923 INFO:teuthology.orchestra.run.smithi143.stdout:(20/28): python3-trustme-0.6.0-4.el8.noarch.rpm 398 kB/s | 27 kB 00:00 2023-09-29T18:21:23.950 INFO:teuthology.orchestra.run.smithi143.stdout:(21/28): python3-waitress-1.4.3-1.el8.noarch.rp 2.9 MB/s | 255 kB 00:00 2023-09-29T18:21:23.965 INFO:teuthology.orchestra.run.smithi143.stdout:(22/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.0 MB/s | 251 kB 00:00 2023-09-29T18:21:23.984 INFO:teuthology.orchestra.run.smithi143.stdout:(23/28): python3-webtest-2.0.33-1.el8.noarch.rp 1.4 MB/s | 85 kB 00:00 2023-09-29T18:21:24.017 INFO:teuthology.orchestra.run.smithi143.stdout:(24/28): python3-zc-lockfile-2.0-2.el8.noarch.r 349 kB/s | 23 kB 00:00 2023-09-29T18:21:24.092 INFO:teuthology.orchestra.run.smithi143.stdout:(25/28): python3-werkzeug-0.12.2-4.el8.noarch.r 3.5 MB/s | 457 kB 00:00 2023-09-29T18:21:24.120 INFO:teuthology.orchestra.run.smithi143.stdout:(26/28): python3-markupsafe-0.23-19.el8.x86_64. 283 kB/s | 39 kB 00:00 2023-09-29T18:21:24.144 INFO:teuthology.orchestra.run.smithi143.stdout:(27/28): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-09-29T18:21:24.213 INFO:teuthology.orchestra.run.smithi143.stdout:(28/28): python3-pytz-2017.2-9.el8.noarch.rpm 450 kB/s | 54 kB 00:00 2023-09-29T18:21:24.214 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:21:24.214 INFO:teuthology.orchestra.run.smithi143.stdout:Total 12 MB/s | 27 MB 00:02 2023-09-29T18:21:24.219 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:21:24.254 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:21:24.254 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:21:24.586 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:21:24.590 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:21:25.615 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:21:25.830 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-09-29T18:21:26.025 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-09-29T18:21:26.181 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-09-29T18:21:26.401 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-09-29T18:21:26.705 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-09-29T18:21:26.944 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-09-29T18:21:27.159 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-09-29T18:21:27.343 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 8/28 2023-09-29T18:21:27.483 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-09-29T18:21:27.606 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-09-29T18:21:27.780 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-09-29T18:21:28.581 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-09-29T18:21:28.709 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-09-29T18:21:28.908 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-09-29T18:21:29.123 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-09-29T18:21:29.325 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-09-29T18:21:29.546 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-09-29T18:21:29.758 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-09-29T18:21:29.953 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-09-29T18:21:30.232 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-09-29T18:21:30.521 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-09-29T18:21:30.783 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-09-29T18:21:31.121 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1143.g2c186684.el8. 23/28 2023-09-29T18:21:31.280 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:17.2.6-1143.g2c186684.el8.x86_64 24/28 2023-09-29T18:21:32.769 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1143.g2c186684.el8.x86_64 24/28 2023-09-29T18:21:32.980 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:17.2.6-1143.g2c186684.el8.x86_64 25/28 2023-09-29T18:21:33.376 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:17.2.6-1143.g2c186684.el8.x86_64 25/28 2023-09-29T18:21:33.493 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:17.2.6-1143.g2c186684.el8.x86_64 26/28 2023-09-29T18:21:33.711 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:17.2.6-1143.g2c186684.el8.x86_64 26/28 2023-09-29T18:21:33.842 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:17.2.6-1143.g2c186684.el8.x86_64 27/28 2023-09-29T18:21:33.860 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:17.2.6-1143.g2c186684.el8.x86_64 27/28 2023-09-29T18:21:33.992 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:17.2.6-1143.g2c186684.el8.x86_64 28/28 2023-09-29T18:21:34.400 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-2:17.2.6-1143.g2c186684.el8.x86_64 28/28 2023-09-29T18:21:34.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:17.2.6-1143.g2c186684.el8.x86_64 1/28 2023-09-29T18:21:34.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:17.2.6-1143.g2c186684.el8.x86_64 2/28 2023-09-29T18:21:34.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:17.2.6-1143.g2c186684.el8.x86_64 3/28 2023-09-29T18:21:34.400 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:17.2.6-1143.g2c186684.el8.x86_64 4/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:17.2.6-1143.g2c186684.el8.x86_64 5/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1143.g2c186684.el8. 6/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-09-29T18:21:34.401 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-09-29T18:21:34.402 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-09-29T18:21:34.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-09-29T18:21:34.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-09-29T18:21:35.946 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-09-29T18:21:35.946 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:26 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:39.434 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:21:39.533 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-29T18:21:39.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-2.6-1.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-29T18:21:39.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:39.536 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:21:39.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-base 2023-09-29T18:21:40.043 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:21:41.721 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:32 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:43.040 INFO:teuthology.orchestra.run.smithi143.stdout:Package ceph-base-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:21:43.528 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:21:43.529 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:21:43.529 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:21:43.648 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install cephadm 2023-09-29T18:21:44.008 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:21:45.339 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:36 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:47.195 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 80 k 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:21:47.196 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:47.197 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 80 k 2023-09-29T18:21:47.197 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 363 k 2023-09-29T18:21:47.197 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:21:47.339 INFO:teuthology.orchestra.run.smithi143.stdout:cephadm-17.2.6-1143.g2c186684.el8.noarch.rpm 561 kB/s | 80 kB 00:00 2023-09-29T18:21:47.339 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:21:47.339 INFO:teuthology.orchestra.run.smithi143.stdout:Total 556 kB/s | 80 kB 00:00 2023-09-29T18:21:47.340 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:21:47.348 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:21:47.348 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:21:47.350 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:21:47.354 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:21:47.405 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:21:50.170 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:21:50.299 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:21:50.504 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:21:51.580 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:21:51.580 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:42 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:54.573 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:21:54.670 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:54.670 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:21:54.670 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:21:54.670 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:54.671 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:21:54.812 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-immutable-object-cache 2023-09-29T18:21:55.187 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:21:56.091 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:47 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:21:57.938 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1143.g2c186684.el8 ceph 161 k 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 161 k 2023-09-29T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 490 k 2023-09-29T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:21:58.304 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-immutable-object-cache-17.2.6-1143.g2c1866 444 kB/s | 161 kB 00:00 2023-09-29T18:21:58.304 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:21:58.304 INFO:teuthology.orchestra.run.smithi143.stdout:Total 442 kB/s | 161 kB 00:00 2023-09-29T18:21:58.305 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:21:58.317 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:21:58.318 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:21:58.370 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:21:58.374 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:21:58.499 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:21:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1143.g2c186684. 1/1 2023-09-29T18:21:59.100 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1143.g2c186684. 1/1 2023-09-29T18:22:00.139 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1143.g2c186684. 1/1 2023-09-29T18:22:00.139 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:51 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:02.249 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:22:02.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:02.346 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:22:02.346 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:22:02.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:02.346 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:02.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr 2023-09-29T18:22:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:04.291 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:55 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:05.657 INFO:teuthology.orchestra.run.smithi143.stdout:Package ceph-mgr-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:22:06.134 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:06.135 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:22:06.136 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:06.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-dashboard 2023-09-29T18:22:06.597 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:07.507 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:58 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:09.366 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout:=================================================================================================== 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout:=================================================================================================== 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 1.7 M 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 23 k 2023-09-29T18:22:09.367 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 14 k 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout:=================================================================================================== 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout:Install 6 Packages 2023-09-29T18:22:09.368 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:09.369 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 2.0 M 2023-09-29T18:22:09.369 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 8.0 M 2023-09-29T18:22:09.369 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:22:09.621 INFO:teuthology.orchestra.run.smithi143.stdout:(1/6): ceph-grafana-dashboards-17.2.6-1143.g2c1 205 kB/s | 23 kB 00:00 2023-09-29T18:22:09.637 INFO:teuthology.orchestra.run.smithi143.stdout:(2/6): ceph-prometheus-alerts-17.2.6-1143.g2c18 113 kB/s | 14 kB 00:00 2023-09-29T18:22:09.764 INFO:teuthology.orchestra.run.smithi143.stdout:(3/6): ceph-mgr-dashboard-17.2.6-1143.g2c186684 6.6 MB/s | 1.7 MB 00:00 2023-09-29T18:22:09.803 INFO:teuthology.orchestra.run.smithi143.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2023-09-29T18:22:09.980 INFO:teuthology.orchestra.run.smithi143.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 94 kB/s | 34 kB 00:00 2023-09-29T18:22:10.167 INFO:teuthology.orchestra.run.smithi143.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 371 kB/s | 196 kB 00:00 2023-09-29T18:22:10.175 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:22:10.175 INFO:teuthology.orchestra.run.smithi143.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2023-09-29T18:22:10.175 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:22:10.180 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:22:10.181 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:22:10.252 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:22:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:22:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:22:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-09-29T18:22:10.945 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-09-29T18:22:11.105 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-09-29T18:22:11.209 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1143.g2c186684.el8.n 4/6 2023-09-29T18:22:11.540 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1143.g2c186684.el8. 5/6 2023-09-29T18:22:11.790 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1143.g2c186684.el8.noarc 6/6 2023-09-29T18:22:12.032 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1143.g2c186684.el8.noarc 6/6 2023-09-29T18:22:12.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1143.g2c186684.el8. 1/6 2023-09-29T18:22:12.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1143.g2c186684.el8.noarc 2/6 2023-09-29T18:22:12.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1143.g2c186684.el8.n 3/6 2023-09-29T18:22:12.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-09-29T18:22:12.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-09-29T18:22:13.050 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-09-29T18:22:13.050 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:04 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:15.294 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-29T18:22:15.395 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-29T18:22:15.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:15.396 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:15.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-diskprediction-local 2023-09-29T18:22:15.873 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:16.746 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:07 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:18.606 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:18.607 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================================================== 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================================================== 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 7.4 M 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-29T18:22:18.608 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================================================== 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout:Install 8 Packages 2023-09-29T18:22:18.609 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:18.610 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 36 M 2023-09-29T18:22:18.610 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 199 M 2023-09-29T18:22:18.610 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:22:18.667 INFO:teuthology.orchestra.run.smithi143.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 3.9 MB/s | 225 kB 00:00 2023-09-29T18:22:18.738 INFO:teuthology.orchestra.run.smithi143.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 53 MB/s | 3.7 MB 00:00 2023-09-29T18:22:18.917 INFO:teuthology.orchestra.run.smithi143.stdout:(3/8): python3-scipy-1.0.0-20.module+el8.1.0+33 47 MB/s | 15 MB 00:00 2023-09-29T18:22:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:(4/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-09-29T18:22:18.952 INFO:teuthology.orchestra.run.smithi143.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 9.4 MB/s | 169 kB 00:00 2023-09-29T18:22:18.976 INFO:teuthology.orchestra.run.smithi143.stdout:(6/8): libgfortran-8.4.1-1.el8.x86_64.rpm 26 MB/s | 639 kB 00:00 2023-09-29T18:22:18.991 INFO:teuthology.orchestra.run.smithi143.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 62 MB/s | 4.6 MB 00:00 2023-09-29T18:22:19.264 INFO:teuthology.orchestra.run.smithi143.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.6-114 11 MB/s | 7.4 MB 00:00 2023-09-29T18:22:19.265 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:22:19.266 INFO:teuthology.orchestra.run.smithi143.stdout:Total 55 MB/s | 36 MB 00:00 2023-09-29T18:22:19.275 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:22:19.294 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:22:19.294 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:22:19.725 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:22:19.729 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:22:20.327 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:22:20.534 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-09-29T18:22:20.678 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-09-29T18:22:20.792 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-09-29T18:22:21.341 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-09-29T18:22:21.517 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-09-29T18:22:22.048 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-09-29T18:22:22.217 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-09-29T18:22:22.949 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-09-29T18:22:23.695 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-09-29T18:22:23.892 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-09-29T18:22:25.949 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-09-29T18:22:27.187 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-09-29T18:22:27.349 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1143.g2c18668 8/8 2023-09-29T18:22:27.652 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1143.g2c18668 8/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1143.g2c18668 1/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-09-29T18:22:27.653 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-09-29T18:22:28.727 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-09-29T18:22:28.727 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:19 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:31.055 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:22:31.155 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.12-1.el8.x86_64 2023-09-29T18:22:31.156 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-09-29T18:22:31.157 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-09-29T18:22:31.157 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-09-29T18:22:31.157 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-09-29T18:22:31.157 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:31.157 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:31.289 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-rook 2023-09-29T18:22:31.654 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:32.563 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:23 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:34.404 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:34.405 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================== 2023-09-29T18:22:34.405 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================== 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 47 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-29T18:22:34.406 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================================== 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout:Install 13 Packages 2023-09-29T18:22:34.407 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:34.408 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 1.9 M 2023-09-29T18:22:34.408 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 25 M 2023-09-29T18:22:34.408 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:22:34.849 INFO:teuthology.orchestra.run.smithi143.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 339 kB/s | 33 kB 00:00 2023-09-29T18:22:34.854 INFO:teuthology.orchestra.run.smithi143.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 159 kB/s | 16 kB 00:00 2023-09-29T18:22:34.884 INFO:teuthology.orchestra.run.smithi143.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 2.3 MB/s | 81 kB 00:00 2023-09-29T18:22:34.903 INFO:teuthology.orchestra.run.smithi143.stdout:(4/13): python3-rsa-4.9-2.el8.noarch.rpm 3.4 MB/s | 64 kB 00:00 2023-09-29T18:22:34.922 INFO:teuthology.orchestra.run.smithi143.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 3.1 MB/s | 61 kB 00:00 2023-09-29T18:22:34.954 INFO:teuthology.orchestra.run.smithi143.stdout:(6/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 848 kB/s | 27 kB 00:00 2023-09-29T18:22:34.971 INFO:teuthology.orchestra.run.smithi143.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.3 MB/s | 126 kB 00:00 2023-09-29T18:22:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 6.0 MB/s | 110 kB 00:00 2023-09-29T18:22:34.990 INFO:teuthology.orchestra.run.smithi143.stdout:(9/13): ceph-mgr-rook-17.2.6-1143.g2c186684.el8 198 kB/s | 47 kB 00:00 2023-09-29T18:22:35.010 INFO:teuthology.orchestra.run.smithi143.stdout:(10/13): python3-jsonpointer-1.10-11.el8.noarch 953 kB/s | 20 kB 00:00 2023-09-29T18:22:35.026 INFO:teuthology.orchestra.run.smithi143.stdout:(11/13): python3-requests-oauthlib-1.0.0-1.el8. 2.7 MB/s | 43 kB 00:00 2023-09-29T18:22:35.028 INFO:teuthology.orchestra.run.smithi143.stdout:(12/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 4.1 MB/s | 155 kB 00:00 2023-09-29T18:22:35.207 INFO:teuthology.orchestra.run.smithi143.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.3 MB/s | 1.2 MB 00:00 2023-09-29T18:22:35.209 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:22:35.210 INFO:teuthology.orchestra.run.smithi143.stdout:Total 2.4 MB/s | 1.9 MB 00:00 2023-09-29T18:22:35.215 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:22:35.229 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:22:35.229 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:22:35.316 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:22:35.321 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:22:35.793 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:22:36.037 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-09-29T18:22:36.297 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/13 2023-09-29T18:22:36.546 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-09-29T18:22:36.814 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-09-29T18:22:37.030 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-09-29T18:22:37.206 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-09-29T18:22:37.387 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-09-29T18:22:37.631 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-09-29T18:22:37.806 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-09-29T18:22:37.999 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-09-29T18:22:39.150 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-09-29T18:22:39.400 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-09-29T18:22:39.514 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:17.2.6-1143.g2c186684.el8.noarch 13/13 2023-09-29T18:22:39.782 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1143.g2c186684.el8.noarch 13/13 2023-09-29T18:22:39.782 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1143.g2c186684.el8.noarch 1/13 2023-09-29T18:22:39.782 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-09-29T18:22:39.783 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-09-29T18:22:39.784 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-09-29T18:22:40.970 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-09-29T18:22:40.970 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:31 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:43.103 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:22:43.201 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-29T18:22:43.202 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-29T18:22:43.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-29T18:22:43.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-29T18:22:43.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-29T18:22:43.203 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:43.203 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:43.325 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-cephadm 2023-09-29T18:22:43.696 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:44.625 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:35 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:46.423 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:46.423 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================================== 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================================== 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 121 k 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-29T18:22:46.424 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================================== 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout:Install 5 Packages 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 6.0 M 2023-09-29T18:22:46.425 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 26 M 2023-09-29T18:22:46.426 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:22:46.892 INFO:teuthology.orchestra.run.smithi143.stdout:(1/5): ceph-mgr-cephadm-17.2.6-1143.g2c186684.e 792 kB/s | 121 kB 00:00 2023-09-29T18:22:47.031 INFO:teuthology.orchestra.run.smithi143.stdout:(2/5): python3-babel-2.5.1-5.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-09-29T18:22:47.145 INFO:teuthology.orchestra.run.smithi143.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 148 kB/s | 60 kB 00:00 2023-09-29T18:22:47.213 INFO:teuthology.orchestra.run.smithi143.stdout:(4/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 2.9 MB/s | 538 kB 00:00 2023-09-29T18:22:47.381 INFO:teuthology.orchestra.run.smithi143.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 741 kB/s | 476 kB 00:00 2023-09-29T18:22:47.385 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:22:47.385 INFO:teuthology.orchestra.run.smithi143.stdout:Total 6.2 MB/s | 6.0 MB 00:00 2023-09-29T18:22:47.388 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:22:47.398 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:22:47.399 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:22:47.489 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:22:47.493 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:22:48.511 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:22:48.863 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-09-29T18:22:49.100 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-09-29T18:22:49.457 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-09-29T18:22:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-09-29T18:22:49.871 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1143.g2c186684.el8.noarch 5/5 2023-09-29T18:22:50.122 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1143.g2c186684.el8.noarch 5/5 2023-09-29T18:22:50.122 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1143.g2c186684.el8.noarch 1/5 2023-09-29T18:22:50.123 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-09-29T18:22:50.123 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-09-29T18:22:50.123 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-09-29T18:22:51.142 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-09-29T18:22:51.142 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:42 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:55.839 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.5.1-5.el8.noarch 2023-09-29T18:22:55.938 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-09-29T18:22:55.939 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-29T18:22:55.939 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:55.939 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:22:56.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-fuse 2023-09-29T18:22:56.417 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:22:57.902 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:48 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:22:59.748 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout: x86_64 2:17.2.6-1143.g2c186684.el8 ceph 874 k 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-09-29T18:22:59.749 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:Install 2 Packages 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 957 k 2023-09-29T18:22:59.750 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 3.0 M 2023-09-29T18:22:59.751 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:22:59.819 INFO:teuthology.orchestra.run.smithi143.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-09-29T18:22:59.967 INFO:teuthology.orchestra.run.smithi143.stdout:(2/2): ceph-fuse-17.2.6-1143.g2c186684.el8.x86_ 3.9 MB/s | 874 kB 00:00 2023-09-29T18:22:59.967 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:22:59.968 INFO:teuthology.orchestra.run.smithi143.stdout:Total 4.3 MB/s | 957 kB 00:00 2023-09-29T18:22:59.969 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:22:59.991 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:22:59.991 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:23:00.072 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:23:00.076 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:23:00.244 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:23:00.464 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-09-29T18:23:01.270 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 2/2 2023-09-29T18:23:01.673 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 2/2 2023-09-29T18:23:01.673 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 1/2 2023-09-29T18:23:02.734 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-09-29T18:23:02.734 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:53 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:08.855 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:23:08.953 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:08.953 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:23:08.953 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-09-29T18:23:08.953 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:08.953 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:09.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-volume 2023-09-29T18:23:09.433 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:11.228 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:02 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:13.105 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:13.106 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:13.106 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:17.2.6-1143.g2c186684.el8 ceph-noarch 267 k 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 267 k 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 1.3 M 2023-09-29T18:23:13.107 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:23:13.283 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-volume-17.2.6-1143.g2c186684.el8.noarch.rp 1.5 MB/s | 267 kB 00:00 2023-09-29T18:23:13.283 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:23:13.284 INFO:teuthology.orchestra.run.smithi143.stdout:Total 1.5 MB/s | 267 kB 00:00 2023-09-29T18:23:13.284 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:23:13.294 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:23:13.294 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:23:13.322 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:23:13.329 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:23:13.784 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:23:14.189 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:23:15.277 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:17.2.6-1143.g2c186684.el8.noarch 1/1 2023-09-29T18:23:16.877 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:06 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:20.600 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:23:20.699 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:20.699 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:23:20.699 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:17.2.6-1143.g2c186684.el8.noarch 2023-09-29T18:23:20.699 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:20.699 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:20.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librados-devel 2023-09-29T18:23:21.197 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:23.006 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:14 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:24.856 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:24.857 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:24.857 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2023-09-29T18:23:24.857 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:24.857 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:23:24.857 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:17.2.6-1143.g2c186684.el8 ceph 130 k 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 130 k 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 464 k 2023-09-29T18:23:24.858 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:23:25.119 INFO:teuthology.orchestra.run.smithi143.stdout:librados-devel-17.2.6-1143.g2c186684.el8.x86_64 497 kB/s | 130 kB 00:00 2023-09-29T18:23:25.120 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:23:25.120 INFO:teuthology.orchestra.run.smithi143.stdout:Total 495 kB/s | 130 kB 00:00 2023-09-29T18:23:25.121 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:23:25.127 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:23:25.127 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:23:25.182 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:23:25.187 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:23:25.308 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:23:25.464 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:25.731 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:26.808 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:26.808 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:17 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:30.732 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:23:30.830 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:30.830 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:23:30.830 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:23:30.830 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:30.830 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:30.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install libcephfs2 2023-09-29T18:23:31.317 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:33.016 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:24 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:34.334 INFO:teuthology.orchestra.run.smithi143.stdout:Package libcephfs2-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:23:34.810 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:23:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:34.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install libcephfs-devel 2023-09-29T18:23:35.289 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:36.884 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:27 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:38.786 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:17.2.6-1143.g2c186684.el8 ceph 25 k 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:38.787 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 25 k 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 109 k 2023-09-29T18:23:38.788 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:23:39.041 INFO:teuthology.orchestra.run.smithi143.stdout:libcephfs-devel-17.2.6-1143.g2c186684.el8.x86_6 98 kB/s | 25 kB 00:00 2023-09-29T18:23:39.042 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:23:39.042 INFO:teuthology.orchestra.run.smithi143.stdout:Total 97 kB/s | 25 kB 00:00 2023-09-29T18:23:39.042 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:23:39.046 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:23:39.046 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:23:39.048 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:23:39.052 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:23:39.108 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:23:39.262 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:39.485 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:40.528 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:23:40.528 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:31 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:44.316 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:23:44.415 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:44.415 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:23:44.415 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:23:44.415 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:23:44.416 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:44.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librados2 2023-09-29T18:23:44.925 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:46.688 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:37 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:48.007 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:23:48.481 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:48.482 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:23:48.482 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:48.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librbd1 2023-09-29T18:23:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:50.723 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:41 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:52.056 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:23:52.531 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:52.532 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:23:52.532 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:52.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rados 2023-09-29T18:23:53.000 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:54.778 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:45 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:23:56.072 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rados-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:23:56.543 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:23:56.544 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:23:56.544 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:23:56.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rgw 2023-09-29T18:23:57.015 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:23:58.835 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:49 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:00.163 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rgw-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:24:00.632 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:00.633 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:24:00.634 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:00.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-cephfs 2023-09-29T18:24:01.113 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:24:03.073 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:54 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:04.446 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-cephfs-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:05.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rbd 2023-09-29T18:24:05.393 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:24:07.184 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:03:58 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:08.504 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rbd-2:17.2.6-1143.g2c186684.el8.x86_64 is already installed. 2023-09-29T18:24:08.972 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:08.973 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-09-29T18:24:08.973 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:09.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-fuse 2023-09-29T18:24:09.445 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:24:11.221 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:02 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:13.316 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:17.2.6-1143.g2c186684.el8 ceph 91 k 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:24:13.317 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:13.318 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:24:13.318 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:13.318 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 91 k 2023-09-29T18:24:13.318 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 264 k 2023-09-29T18:24:13.318 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:24:13.583 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-fuse-17.2.6-1143.g2c186684.el8.x86_64.rpm 342 kB/s | 91 kB 00:00 2023-09-29T18:24:13.583 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:24:13.584 INFO:teuthology.orchestra.run.smithi143.stdout:Total 340 kB/s | 91 kB 00:00 2023-09-29T18:24:13.584 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:24:13.596 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:24:13.596 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:24:13.651 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:24:13.655 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:24:13.770 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:24:13.965 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:14.239 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:15.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:15.263 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:06 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:17.188 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:24:17.286 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:17.286 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:24:17.286 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:24:17.286 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:17.286 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:17.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-mirror 2023-09-29T18:24:17.772 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:24:18.629 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:09 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:20.417 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:20.417 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:17.2.6-1143.g2c186684.el8 ceph 3.5 M 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:24:20.418 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:20.419 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 3.5 M 2023-09-29T18:24:20.419 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 16 M 2023-09-29T18:24:20.419 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:24:20.739 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-mirror-17.2.6-1143.g2c186684.el8.x86_64.rpm 11 MB/s | 3.5 MB 00:00 2023-09-29T18:24:20.740 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:24:20.741 INFO:teuthology.orchestra.run.smithi143.stdout:Total 11 MB/s | 3.5 MB 00:00 2023-09-29T18:24:20.741 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:24:20.754 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:24:20.755 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:24:20.823 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:24:20.828 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:24:21.276 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:24:21.448 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:21.861 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:22.887 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:22.888 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:13 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:24.815 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:24:24.913 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:24.913 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:24:24.913 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:24:24.913 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:24.913 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:25.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-nbd 2023-09-29T18:24:25.402 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-09-29T18:24:26.284 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:17 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:28.138 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:17.2.6-1143.g2c186684.el8 ceph 173 k 2023-09-29T18:24:28.139 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 173 k 2023-09-29T18:24:28.140 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 552 k 2023-09-29T18:24:28.141 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-09-29T18:24:28.743 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-nbd-17.2.6-1143.g2c186684.el8.x86_64.rpm 287 kB/s | 173 kB 00:00 2023-09-29T18:24:28.743 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-09-29T18:24:28.744 INFO:teuthology.orchestra.run.smithi143.stdout:Total 286 kB/s | 173 kB 00:00 2023-09-29T18:24:28.744 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-09-29T18:24:28.758 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-09-29T18:24:28.758 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-09-29T18:24:28.816 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-09-29T18:24:28.820 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-09-29T18:24:28.950 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-09-29T18:24:29.112 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:29.389 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-nbd-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:30.414 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:17.2.6-1143.g2c186684.el8.x86_64 1/1 2023-09-29T18:24:30.415 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:04:21 ago on Fri 29 Sep 2023 06:20:09 PM UTC. 2023-09-29T18:24:32.390 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-09-29T18:24:32.488 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-09-29T18:24:32.488 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:17.2.6-1143.g2c186684.el8.x86_64 2023-09-29T18:24:32.488 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:32.488 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-09-29T18:24:32.610 DEBUG:teuthology.parallel:result is None 2023-09-29T18:24:32.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2c186684db7f9ea0cdb098fc86b350a0fdd26eba 2023-09-29T18:24:32.836 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-29T18:24:32.856 INFO:teuthology.orchestra.run.smithi143.stdout:17.2.6-1143.g2c186684.el8 2023-09-29T18:24:32.856 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1143.g2c186684.el8 2023-09-29T18:24:32.856 INFO:teuthology.task.install:The correct ceph version 17.2.6-1143.g2c186684 is installed. 2023-09-29T18:24:32.856 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-29T18:24:32.857 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:32.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-29T18:24:32.922 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-29T18:24:32.922 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:32.922 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2023-09-29T18:24:32.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-29T18:24:33.051 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-29T18:24:33.051 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:33.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-29T18:24:33.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-29T18:24:33.182 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-29T18:24:33.183 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:33.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2023-09-29T18:24:33.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-29T18:24:33.311 INFO:teuthology.run_tasks:Running task ceph... 2023-09-29T18:24:33.381 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-29T18:24:33.381 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2023-09-29T18:24:33.405 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-29T18:24:33.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-29T18:24:33.469 INFO:tasks.ceph:Creating extra log directories... 2023-09-29T18:24:33.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-29T18:24:33.541 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-29T18:24:33.541 INFO:tasks.ceph:config {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-29T18:24:33.541 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834', 'branch': 'wip-yuri3-testing-2023-09-27-0919-quincy', 'description': 'rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic}', 'email': None, 'first_in_suite': False, 'job_id': '7405834', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-09-27-0919-quincy'}, 'ceph': {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-09-27-0919-quincy', 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 1925, 'sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri3-testing-2023-09-27-0919-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '2c186684db7f9ea0cdb098fc86b350a0fdd26eba', 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1dF7bKZeldZ/9Dl+g0zDsW2d5rAyu7pDHG6+AZnv2ETsbs1GwvhTB0ctyT6q5rNSs11+G17pq8WTLQX3HB918='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'workunit': {'clients': {'client.0': ['rbd/cli_generic.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-28_23:00:03', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-29T18:24:33.542 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-29T18:24:33.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2023-09-29T18:24:33.653 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:33.653 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2023-09-29T18:24:33.708 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-29T18:24:33.708 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 215732 Links: 1 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-09-29T18:24:33.763 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 18:24:21.627035801 +0000 2023-09-29T18:24:33.764 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 18:19:27.210150071 +0000 2023-09-29T18:24:33.764 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 18:19:27.210150071 +0000 2023-09-29T18:24:33.764 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T18:24:33.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-29T18:24:33.826 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T18:24:33.826 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T18:24:33.826 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000255162 s, 2.0 MB/s 2023-09-29T18:24:33.827 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-29T18:24:33.883 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2023-09-29T18:24:33.938 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-29T18:24:33.938 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T18:24:33.938 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 216535 Links: 1 2023-09-29T18:24:33.938 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T18:24:33.939 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-09-29T18:24:33.939 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 18:24:21.627035801 +0000 2023-09-29T18:24:33.939 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 18:19:27.701145882 +0000 2023-09-29T18:24:33.939 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 18:19:27.701145882 +0000 2023-09-29T18:24:33.939 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T18:24:33.939 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-29T18:24:34.001 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T18:24:34.001 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T18:24:34.001 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000233236 s, 2.2 MB/s 2023-09-29T18:24:34.002 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-29T18:24:34.058 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 216641 Links: 1 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-09-29T18:24:34.113 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 18:24:21.627035801 +0000 2023-09-29T18:24:34.114 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 18:19:28.193141685 +0000 2023-09-29T18:24:34.114 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 18:19:28.193141685 +0000 2023-09-29T18:24:34.114 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T18:24:34.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-29T18:24:34.176 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T18:24:34.176 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T18:24:34.176 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000221969 s, 2.3 MB/s 2023-09-29T18:24:34.177 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-29T18:24:34.232 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2023-09-29T18:24:34.287 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 216703 Links: 1 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 18:24:21.627035801 +0000 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 18:19:28.677137557 +0000 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 18:19:28.677137557 +0000 2023-09-29T18:24:34.288 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T18:24:34.289 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-29T18:24:34.351 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T18:24:34.351 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T18:24:34.351 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000194256 s, 2.6 MB/s 2023-09-29T18:24:34.352 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-29T18:24:34.408 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-09-29T18:24:34.408 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-09-29T18:24:34.408 INFO:tasks.ceph:Generating config... 2023-09-29T18:24:34.409 INFO:tasks.ceph:[client] rbd default format = 1 2023-09-29T18:24:34.409 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-09-29T18:24:34.409 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-09-29T18:24:34.409 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-09-29T18:24:34.410 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-29T18:24:34.410 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-29T18:24:34.410 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-29T18:24:34.410 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-29T18:24:34.410 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-29T18:24:34.410 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-29T18:24:34.410 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-29T18:24:34.410 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2023-09-29T18:24:34.410 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-29T18:24:34.410 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-29T18:24:34.411 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-09-29T18:24:34.411 INFO:tasks.ceph:Setting up mon.a... 2023-09-29T18:24:34.412 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-29T18:24:34.503 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2023-09-29T18:24:34.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-29T18:24:34.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-29T18:24:34.609 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143')] 2023-09-29T18:24:34.609 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default format': 1}, 'mon.a': {}} 2023-09-29T18:24:34.610 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:34.610 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-29T18:24:34.664 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --print /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:34.738 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid 210a5970-1bec-4387-ae0f-c041593b74fd 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = octopus 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:fsid 210a5970-1bec-4387-ae0f-c041593b74fd 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2023-09-29T18:24:34.739914+0000 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:created 2023-09-29T18:24:34.739914+0000 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 15 (octopus) 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 1 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2023-09-29T18:24:34.739 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-09-29T18:24:34.740 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-29T18:24:34.795 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 210a5970-1bec-4387-ae0f-c041593b74fd... 2023-09-29T18:24:34.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-09-29T18:24:34.878 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.879 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-09-29T18:24:34.880 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-09-29T18:24:34.881 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = 172.21.15.143 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: mon client directed command retry = 5 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on pool no app = False 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: ms inject socket failures = 5000 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 210a5970-1bec-4387-ae0f-c041593b74fd 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-09-29T18:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-09-29T18:24:34.883 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-09-29T18:24:34.884 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = stupid 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-09-29T18:24:34.885 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: osd shutdown pgref assert = True 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-09-29T18:24:34.886 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-09-29T18:24:34.887 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-29T18:24:34.888 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-09-29T18:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-09-29T18:24:34.890 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-29T18:24:34.890 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-29T18:24:34.890 INFO:teuthology.orchestra.run.smithi143.stdout: rbd default format = 1 2023-09-29T18:24:34.890 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-09-29T18:24:34.891 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-29T18:24:34.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-29T18:24:34.982 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-29T18:24:34.982 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:34.982 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-29T18:24:34.996 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:34.997 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-29T18:24:35.050 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:24:35.051 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:35.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-29T18:24:35.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-29T18:24:35.124 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:35.124 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:35.179 INFO:tasks.ceph:Setting up mon nodes... 2023-09-29T18:24:35.179 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-29T18:24:35.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-29T18:24:35.267 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-29T18:24:35.269 INFO:tasks.ceph:Setting up mds nodes... 2023-09-29T18:24:35.269 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-29T18:24:35.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-29T18:24:35.307 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-29T18:24:35.319 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-29T18:24:35.319 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-09-29T18:24:35.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-29T18:24:35.383 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-09-29T18:24:35.383 INFO:tasks.ceph:role: osd.0 2023-09-29T18:24:35.384 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:24:35.384 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-29T18:24:35.454 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T18:24:35.455 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T18:24:35.884 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-09-29T18:24:35.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-09-29T18:24:35.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-29T18:24:36.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-29T18:24:36.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-29T18:24:36.145 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-09-29T18:24:36.145 INFO:tasks.ceph:role: osd.1 2023-09-29T18:24:36.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:24:36.146 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-29T18:24:36.219 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T18:24:36.220 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T18:24:36.637 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-09-29T18:24:36.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-09-29T18:24:36.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-29T18:24:36.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-29T18:24:36.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-29T18:24:36.842 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-09-29T18:24:36.842 INFO:tasks.ceph:role: osd.2 2023-09-29T18:24:36.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:24:36.843 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T18:24:36.914 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T18:24:37.363 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-09-29T18:24:37.365 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-09-29T18:24:37.365 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-29T18:24:37.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-29T18:24:37.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:37.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.657+0000 7f09b93eb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-29T18:24:37.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.661+0000 7f09b93eb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-29T18:24:37.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.661+0000 7f09b93eb540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T18:24:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.661+0000 7f09b93eb540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T18:24:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.661+0000 7f09b93eb540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T18:24:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:37.662+0000 7f09b93eb540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-29T18:24:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:38.188+0000 7f09b93eb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:38.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:38.958+0000 7f09b93eb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:39.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:39.729+0000 7f09b93eb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:40.527 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-29T18:24:40.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T18:24:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:40.638+0000 7f02249e3540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-29T18:24:41.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:41.178+0000 7f02249e3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:41.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:41.949+0000 7f02249e3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:42.710+0000 7f02249e3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:43.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-29T18:24:43.583 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.664+0000 7f16d00dc540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-29T18:24:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.664+0000 7f16d00dc540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-29T18:24:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.664+0000 7f16d00dc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T18:24:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.664+0000 7f16d00dc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T18:24:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.664+0000 7f16d00dc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T18:24:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:43.665+0000 7f16d00dc540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-29T18:24:44.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:44.183+0000 7f16d00dc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:44.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:44.955+0000 7f16d00dc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:45.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T18:24:45.721+0000 7f16d00dc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-29T18:24:46.577 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-29T18:24:46.577 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:46.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-29T18:24:46.641 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:46.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-29T18:24:46.705 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:46.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-29T18:24:46.769 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:46.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-29T18:24:46.833 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:46.833 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-29T18:24:46.888 INFO:tasks.ceph:Adding keys to all mons... 2023-09-29T18:24:46.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBjFhdl2A37DxAAmcc3Ve+3h4NeEOl2Bp3a/Q== 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBlFhdlIGlHJxAAXn9DCvqzjTk1tFHh8kALLg== 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBoFhdlnF4eJhAARDEIuTqcssXJIXdLWsrxFQ== 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2023-09-29T18:24:46.951 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBrFhdley2vJxAApBW6nT4CXqhIXBTlSNZeXw== 2023-09-29T18:24:46.952 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-09-29T18:24:46.952 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBjFhdlf5NeEhAAs2/aW7shMVjIdmBiYwstJA== 2023-09-29T18:24:46.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T18:24:47.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T18:24:47.086 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T18:24:47.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T18:24:47.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-29T18:24:47.229 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-29T18:24:47.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-29T18:24:47.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-29T18:24:47.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-29T18:24:47.368 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-29T18:24:47.422 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-29T18:24:47.423 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-29T18:24:47.423 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-29T18:24:47.464 INFO:tasks.ceph.mon.a:Started 2023-09-29T18:24:47.465 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-29T18:24:47.465 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-29T18:24:47.465 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-29T18:24:47.467 INFO:tasks.ceph.mgr.x:Started 2023-09-29T18:24:47.467 INFO:tasks.ceph:Setting crush tunables to default 2023-09-29T18:24:47.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2023-09-29T18:24:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2023-09-29T18:24:47.688 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-29T18:24:47.688 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:47.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-29T18:24:47.712 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:47.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-29T18:24:47.776 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:24:47.776 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-29T18:24:47.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new d8bf0b0d-c569-4438-91c9-9604198a9a14 0 2023-09-29T18:24:48.116 INFO:teuthology.orchestra.run.smithi143.stdout:0 2023-09-29T18:24:48.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new f208ee2f-e70e-4067-862e-5eb63df7af24 1 2023-09-29T18:24:48.334 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-09-29T18:24:48.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new ade283d5-0932-421f-a080-6c4a679eaf37 2 2023-09-29T18:24:48.555 INFO:teuthology.orchestra.run.smithi143.stdout:2 2023-09-29T18:24:48.567 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-29T18:24:48.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-29T18:24:48.569 INFO:tasks.ceph.osd.0:Started 2023-09-29T18:24:48.570 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-29T18:24:48.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-29T18:24:48.572 INFO:tasks.ceph.osd.1:Started 2023-09-29T18:24:48.572 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-29T18:24:48.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-29T18:24:48.574 INFO:tasks.ceph.osd.2:Started 2023-09-29T18:24:48.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T18:24:48.796 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:24:48.796 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:24:48.553467+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:24:48.809 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-29T18:24:48.809 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-29T18:24:48.982 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:48.982+0000 7ff6a7f5b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-29T18:24:49.111 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T18:24:49.117 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.116+0000 7ff6a7f5b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-29T18:24:49.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:24:49.181+0000 7f1746e4c540 -1 Falling back to public interface 2023-09-29T18:24:49.195 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.195+0000 7ff6a7f5b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-29T18:24:49.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:24:49.198+0000 7f306b74b540 -1 Falling back to public interface 2023-09-29T18:24:49.211 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-09-29T18:24:49.210+0000 7f9a91cf5540 -1 Falling back to public interface 2023-09-29T18:24:49.298 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T18:24:49.298 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":5,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:24:48.553467+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:24:49.309 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-09-29T18:24:49.389 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.389+0000 7ff6a7f5b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-29T18:24:49.578 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.577+0000 7ff6a7f5b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-29T18:24:49.656 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.655+0000 7ff6a7f5b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-29T18:24:49.795 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.794+0000 7ff6a7f5b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-29T18:24:49.892 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:49.892+0000 7ff6a7f5b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-29T18:24:50.423 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:50.422+0000 7ff6a7f5b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-29T18:24:50.502 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:50.502+0000 7ff6a7f5b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-29T18:24:50.581 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:50.581+0000 7ff6a7f5b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-29T18:24:51.174 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:51.170+0000 7ff6a7f5b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-29T18:24:51.312 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:51.311+0000 7ff6a7f5b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-29T18:24:51.405 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:51.404+0000 7ff6a7f5b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-29T18:24:51.662 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:51.662+0000 7ff6a7f5b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-29T18:24:51.743 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:51.742+0000 7ff6a7f5b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-29T18:24:52.634 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:52.634+0000 7ff6a7f5b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-29T18:24:52.713 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:52.713+0000 7ff6a7f5b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-29T18:24:52.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:24:52.724+0000 7f1746e4c540 -1 osd.1 0 log_to_monitors true 2023-09-29T18:24:52.788 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-09-29T18:24:52.788+0000 7f9a91cf5540 -1 osd.2 0 log_to_monitors true 2023-09-29T18:24:52.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:24:52.795+0000 7f306b74b540 -1 osd.0 0 log_to_monitors true 2023-09-29T18:24:52.803 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:52.803+0000 7ff6a7f5b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-29T18:24:52.940 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:52.940+0000 7ff6a7f5b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-29T18:24:53.027 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:53.026+0000 7ff6a7f5b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-29T18:24:53.183 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:53.183+0000 7ff6a7f5b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-29T18:24:53.263 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:53.263+0000 7ff6a7f5b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-29T18:24:53.341 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:53.340+0000 7ff6a7f5b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-29T18:24:53.418 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:53.417+0000 7ff6a7f5b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-29T18:24:55.611 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T18:24:55.873 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T18:24:55.874 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":7,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:24:54.554401+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:24:55.887 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-09-29T18:24:56.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:24:56.281+0000 7f306236c700 -1 osd.0 0 waiting for initial osdmap 2023-09-29T18:24:56.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:24:56.290+0000 7f305b163700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T18:24:56.292 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-09-29T18:24:56.292+0000 7f9a8a119700 -1 osd.2 0 waiting for initial osdmap 2023-09-29T18:24:56.300 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-09-29T18:24:56.300+0000 7f9a8170d700 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T18:24:56.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:24:56.390+0000 7f173f270700 -1 osd.1 0 waiting for initial osdmap 2023-09-29T18:24:56.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:24:56.397+0000 7f1736864700 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T18:24:56.519 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T18:24:56.518+0000 7ff66135e700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-09-29T18:25:02.190 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T18:25:02.496 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T18:25:02.496 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":11,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:25:00.524679+0000","last_up_change":"2023-09-29T18:24:57.285879+0000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T18:24:57.558720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":86744},{"type":"v1","addr":"172.21.15.143:6809","nonce":86744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":86744},{"type":"v1","addr":"172.21.15.143:6811","nonce":86744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":86744},{"type":"v1","addr":"172.21.15.143:6815","nonce":86744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":86744},{"type":"v1","addr":"172.21.15.143:6813","nonce":86744}]},"public_addr":"172.21.15.143:6809/86744","cluster_addr":"172.21.15.143:6811/86744","heartbeat_back_addr":"172.21.15.143:6815/86744","heartbeat_front_addr":"172.21.15.143:6813/86744","state":["exists","up"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":86743},{"type":"v1","addr":"172.21.15.143:6801","nonce":86743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":86743},{"type":"v1","addr":"172.21.15.143:6803","nonce":86743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":86743},{"type":"v1","addr":"172.21.15.143:6807","nonce":86743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":86743},{"type":"v1","addr":"172.21.15.143:6805","nonce":86743}]},"public_addr":"172.21.15.143:6801/86743","cluster_addr":"172.21.15.143:6803/86743","heartbeat_back_addr":"172.21.15.143:6807/86743","heartbeat_front_addr":"172.21.15.143:6805/86743","state":["exists","up"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":86759},{"type":"v1","addr":"172.21.15.143:6817","nonce":86759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":86759},{"type":"v1","addr":"172.21.15.143:6819","nonce":86759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":86759},{"type":"v1","addr":"172.21.15.143:6823","nonce":86759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":86759},{"type":"v1","addr":"172.21.15.143:6821","nonce":86759}]},"public_addr":"172.21.15.143:6817/86759","cluster_addr":"172.21.15.143:6819/86759","heartbeat_back_addr":"172.21.15.143:6823/86759","heartbeat_front_addr":"172.21.15.143:6821/86759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.827743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.725152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.753719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:25:02.508 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-09-29T18:25:02.509 INFO:tasks.ceph:Creating RBD pool 2023-09-29T18:25:02.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-29T18:25:03.577 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'rbd' created 2023-09-29T18:25:03.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-29T18:25:04.581 INFO:teuthology.orchestra.run.smithi143.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-29T18:25:04.593 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-29T18:25:04.593 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-29T18:25:04.594 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-29T18:25:04.594 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-29T18:25:04.594 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-29T18:25:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:04.966 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":86405},{"type":"v1","addr":"172.21.15.143:6825","nonce":86405}]},"active_addr":"172.21.15.143:6825/86405","active_change":"2023-09-29T18:24:55.443403+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3855365400}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3209616106}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3140865523}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3275659122}]}]} 2023-09-29T18:25:04.969 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-29T18:25:04.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-29T18:25:04.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T18:25:05.274 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:05.274 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":13,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:25:04.566872+0000","last_up_change":"2023-09-29T18:24:57.285879+0000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T18:24:57.558720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T18:25:02.831861+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":86744},{"type":"v1","addr":"172.21.15.143:6809","nonce":86744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":86744},{"type":"v1","addr":"172.21.15.143:6811","nonce":86744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":86744},{"type":"v1","addr":"172.21.15.143:6815","nonce":86744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":86744},{"type":"v1","addr":"172.21.15.143:6813","nonce":86744}]},"public_addr":"172.21.15.143:6809/86744","cluster_addr":"172.21.15.143:6811/86744","heartbeat_back_addr":"172.21.15.143:6815/86744","heartbeat_front_addr":"172.21.15.143:6813/86744","state":["exists","up"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":86743},{"type":"v1","addr":"172.21.15.143:6801","nonce":86743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":86743},{"type":"v1","addr":"172.21.15.143:6803","nonce":86743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":86743},{"type":"v1","addr":"172.21.15.143:6807","nonce":86743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":86743},{"type":"v1","addr":"172.21.15.143:6805","nonce":86743}]},"public_addr":"172.21.15.143:6801/86743","cluster_addr":"172.21.15.143:6803/86743","heartbeat_back_addr":"172.21.15.143:6807/86743","heartbeat_front_addr":"172.21.15.143:6805/86743","state":["exists","up"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":86759},{"type":"v1","addr":"172.21.15.143:6817","nonce":86759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":86759},{"type":"v1","addr":"172.21.15.143:6819","nonce":86759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":86759},{"type":"v1","addr":"172.21.15.143:6823","nonce":86759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":86759},{"type":"v1","addr":"172.21.15.143:6821","nonce":86759}]},"public_addr":"172.21.15.143:6817/86759","cluster_addr":"172.21.15.143:6819/86759","heartbeat_back_addr":"172.21.15.143:6823/86759","heartbeat_front_addr":"172.21.15.143:6821/86759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.827743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.725152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.753719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:25:05.288 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-29T18:25:05.288 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T18:25:05.597 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:05.598 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":14,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T18:25:05.574358+0000","last_up_change":"2023-09-29T18:24:57.285879+0000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T18:24:57.558720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T18:25:02.831861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":86744},{"type":"v1","addr":"172.21.15.143:6809","nonce":86744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":86744},{"type":"v1","addr":"172.21.15.143:6811","nonce":86744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":86744},{"type":"v1","addr":"172.21.15.143:6815","nonce":86744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":86744},{"type":"v1","addr":"172.21.15.143:6813","nonce":86744}]},"public_addr":"172.21.15.143:6809/86744","cluster_addr":"172.21.15.143:6811/86744","heartbeat_back_addr":"172.21.15.143:6815/86744","heartbeat_front_addr":"172.21.15.143:6813/86744","state":["exists","up"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":86743},{"type":"v1","addr":"172.21.15.143:6801","nonce":86743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":86743},{"type":"v1","addr":"172.21.15.143:6803","nonce":86743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":86743},{"type":"v1","addr":"172.21.15.143:6807","nonce":86743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":86743},{"type":"v1","addr":"172.21.15.143:6805","nonce":86743}]},"public_addr":"172.21.15.143:6801/86743","cluster_addr":"172.21.15.143:6803/86743","heartbeat_back_addr":"172.21.15.143:6807/86743","heartbeat_front_addr":"172.21.15.143:6805/86743","state":["exists","up"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":86759},{"type":"v1","addr":"172.21.15.143:6817","nonce":86759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":86759},{"type":"v1","addr":"172.21.15.143:6819","nonce":86759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":86759},{"type":"v1","addr":"172.21.15.143:6823","nonce":86759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":86759},{"type":"v1","addr":"172.21.15.143:6821","nonce":86759}]},"public_addr":"172.21.15.143:6817/86759","cluster_addr":"172.21.15.143:6819/86759","heartbeat_back_addr":"172.21.15.143:6823/86759","heartbeat_front_addr":"172.21.15.143:6821/86759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.827743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.725152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.753719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T18:25:05.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-29T18:25:05.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-29T18:25:05.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-29T18:25:05.769 INFO:teuthology.orchestra.run.smithi143.stdout:34359738372 2023-09-29T18:25:05.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-29T18:25:05.792 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:05.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-29T18:25:05.793 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:05.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-29T18:25:06.107 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:06.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2023-09-29T18:25:06.158 INFO:teuthology.orchestra.run.smithi143.stdout:34359738370 2023-09-29T18:25:06.168 INFO:teuthology.orchestra.run.smithi143.stdout:34359738370 2023-09-29T18:25:06.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.1 2023-09-29T18:25:06.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.2 2023-09-29T18:25:07.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-29T18:25:07.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-29T18:25:07.186 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-29T18:25:07.451 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:07.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2023-09-29T18:25:07.505 INFO:teuthology.orchestra.run.smithi143.stdout:34359738370 2023-09-29T18:25:07.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.1 2023-09-29T18:25:07.545 INFO:teuthology.orchestra.run.smithi143.stdout:34359738370 2023-09-29T18:25:07.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.2 2023-09-29T18:25:08.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-29T18:25:08.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-29T18:25:08.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-29T18:25:08.837 INFO:teuthology.orchestra.run.smithi143.stdout:34359738372 2023-09-29T18:25:08.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2023-09-29T18:25:08.852 DEBUG:teuthology.parallel:result is None 2023-09-29T18:25:08.865 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:08.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738371 for osd.1 2023-09-29T18:25:08.880 DEBUG:teuthology.parallel:result is None 2023-09-29T18:25:08.899 INFO:teuthology.orchestra.run.smithi143.stdout:34359738371 2023-09-29T18:25:08.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738371 for osd.2 2023-09-29T18:25:08.913 DEBUG:teuthology.parallel:result is None 2023-09-29T18:25:08.913 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-29T18:25:08.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T18:25:09.256 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T18:25:09.269 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-09-29T18:25:07.519150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25144,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":23616,"kb_avail":283090376,"statfs":{"total":289910292480,"available":289884545024,"internally_reserved":0,"allocated":1540096,"data_stored":1277679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24182784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000714"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586747+0000","last_change":"2023-09-29T18:25:04.585474+0000","last_active":"2023-09-29T18:25:04.586747+0000","last_peered":"2023-09-29T18:25:04.586747+0000","last_clean":"2023-09-29T18:25:04.586747+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T18:25:04.586747+0000","last_undegraded":"2023-09-29T18:25:04.586747+0000","last_fullsized":"2023-09-29T18:25:04.586747+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586701+0000","last_change":"2023-09-29T18:25:04.585353+0000","last_active":"2023-09-29T18:25:04.586701+0000","last_peered":"2023-09-29T18:25:04.586701+0000","last_clean":"2023-09-29T18:25:04.586701+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T18:25:04.586701+0000","last_undegraded":"2023-09-29T18:25:04.586701+0000","last_fullsized":"2023-09-29T18:25:04.586701+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586793+0000","last_change":"2023-09-29T18:25:04.585430+0000","last_active":"2023-09-29T18:25:04.586793+0000","last_peered":"2023-09-29T18:25:04.586793+0000","last_clean":"2023-09-29T18:25:04.586793+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T18:25:04.586793+0000","last_undegraded":"2023-09-29T18:25:04.586793+0000","last_fullsized":"2023-09-29T18:25:04.586793+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.587200+0000","last_change":"2023-09-29T18:25:04.587200+0000","last_active":"2023-09-29T18:25:04.587200+0000","last_peered":"2023-09-29T18:25:04.587200+0000","last_clean":"2023-09-29T18:25:04.587200+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T18:25:04.587200+0000","last_undegraded":"2023-09-29T18:25:04.587200+0000","last_fullsized":"2023-09-29T18:25:04.587200+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.579538+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T18:25:04.579538+0000","last_peered":"2023-09-29T18:25:04.579538+0000","last_clean":"2023-09-29T18:25:04.579538+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T18:25:04.579538+0000","last_undegraded":"2023-09-29T18:25:04.579538+0000","last_fullsized":"2023-09-29T18:25:04.579538+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.587988+0000","last_change":"2023-09-29T18:25:04.586851+0000","last_active":"2023-09-29T18:25:04.587988+0000","last_peered":"2023-09-29T18:25:04.587988+0000","last_clean":"2023-09-29T18:25:04.587988+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T18:25:04.587988+0000","last_undegraded":"2023-09-29T18:25:04.587988+0000","last_fullsized":"2023-09-29T18:25:04.587988+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.585269+0000","last_change":"2023-09-29T18:25:04.585269+0000","last_active":"2023-09-29T18:25:04.585269+0000","last_peered":"2023-09-29T18:25:04.585269+0000","last_clean":"2023-09-29T18:25:04.585269+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T18:25:04.585269+0000","last_undegraded":"2023-09-29T18:25:04.585269+0000","last_fullsized":"2023-09-29T18:25:04.585269+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.585932+0000","last_change":"2023-09-29T18:25:04.585932+0000","last_active":"2023-09-29T18:25:04.585932+0000","last_peered":"2023-09-29T18:25:04.585932+0000","last_clean":"2023-09-29T18:25:04.585932+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T18:25:04.585932+0000","last_undegraded":"2023-09-29T18:25:04.585932+0000","last_fullsized":"2023-09-29T18:25:04.585932+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.587359+0000","last_change":"2023-09-29T18:25:04.585927+0000","last_active":"2023-09-29T18:25:04.587359+0000","last_peered":"2023-09-29T18:25:04.587359+0000","last_clean":"2023-09-29T18:25:04.587359+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T18:25:04.587359+0000","last_undegraded":"2023-09-29T18:25:04.587359+0000","last_fullsized":"2023-09-29T18:25:04.587359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7736,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364104,"statfs":{"total":96636764160,"available":96628842496,"internally_reserved":0,"allocated":114688,"data_stored":31159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8700,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363140,"statfs":{"total":96636764160,"available":96627855360,"internally_reserved":0,"allocated":708608,"data_stored":621527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8708,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363132,"statfs":{"total":96636764160,"available":96627847168,"internally_reserved":0,"allocated":716800,"data_stored":624993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T18:25:09.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T18:25:09.573 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T18:25:09.586 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-09-29T18:25:09.519495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25144,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":23616,"kb_avail":283090376,"statfs":{"total":289910292480,"available":289884545024,"internally_reserved":0,"allocated":1540096,"data_stored":1277679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24182784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001059"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586747+0000","last_change":"2023-09-29T18:25:04.585474+0000","last_active":"2023-09-29T18:25:04.586747+0000","last_peered":"2023-09-29T18:25:04.586747+0000","last_clean":"2023-09-29T18:25:04.586747+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T18:25:04.586747+0000","last_undegraded":"2023-09-29T18:25:04.586747+0000","last_fullsized":"2023-09-29T18:25:04.586747+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586701+0000","last_change":"2023-09-29T18:25:04.585353+0000","last_active":"2023-09-29T18:25:04.586701+0000","last_peered":"2023-09-29T18:25:04.586701+0000","last_clean":"2023-09-29T18:25:04.586701+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T18:25:04.586701+0000","last_undegraded":"2023-09-29T18:25:04.586701+0000","last_fullsized":"2023-09-29T18:25:04.586701+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586793+0000","last_change":"2023-09-29T18:25:04.585430+0000","last_active":"2023-09-29T18:25:04.586793+0000","last_peered":"2023-09-29T18:25:04.586793+0000","last_clean":"2023-09-29T18:25:04.586793+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T18:25:04.586793+0000","last_undegraded":"2023-09-29T18:25:04.586793+0000","last_fullsized":"2023-09-29T18:25:04.586793+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.590219+0000","last_change":"2023-09-29T18:25:04.587200+0000","last_active":"2023-09-29T18:25:04.590219+0000","last_peered":"2023-09-29T18:25:04.590219+0000","last_clean":"2023-09-29T18:25:04.590219+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T18:25:04.590219+0000","last_undegraded":"2023-09-29T18:25:04.590219+0000","last_fullsized":"2023-09-29T18:25:04.590219+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.579538+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T18:25:04.579538+0000","last_peered":"2023-09-29T18:25:04.579538+0000","last_clean":"2023-09-29T18:25:04.579538+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T18:25:04.579538+0000","last_undegraded":"2023-09-29T18:25:04.579538+0000","last_fullsized":"2023-09-29T18:25:04.579538+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.587988+0000","last_change":"2023-09-29T18:25:04.586851+0000","last_active":"2023-09-29T18:25:04.587988+0000","last_peered":"2023-09-29T18:25:04.587988+0000","last_clean":"2023-09-29T18:25:04.587988+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T18:25:04.587988+0000","last_undegraded":"2023-09-29T18:25:04.587988+0000","last_fullsized":"2023-09-29T18:25:04.587988+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.586980+0000","last_change":"2023-09-29T18:25:04.585269+0000","last_active":"2023-09-29T18:25:04.586980+0000","last_peered":"2023-09-29T18:25:04.586980+0000","last_clean":"2023-09-29T18:25:04.586980+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T18:25:04.586980+0000","last_undegraded":"2023-09-29T18:25:04.586980+0000","last_fullsized":"2023-09-29T18:25:04.586980+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.589631+0000","last_change":"2023-09-29T18:25:04.585932+0000","last_active":"2023-09-29T18:25:04.589631+0000","last_peered":"2023-09-29T18:25:04.589631+0000","last_clean":"2023-09-29T18:25:04.589631+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T18:25:04.589631+0000","last_undegraded":"2023-09-29T18:25:04.589631+0000","last_fullsized":"2023-09-29T18:25:04.589631+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-09-29T18:25:04.587359+0000","last_change":"2023-09-29T18:25:04.585927+0000","last_active":"2023-09-29T18:25:04.587359+0000","last_peered":"2023-09-29T18:25:04.587359+0000","last_clean":"2023-09-29T18:25:04.587359+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T18:25:04.587359+0000","last_undegraded":"2023-09-29T18:25:04.587359+0000","last_fullsized":"2023-09-29T18:25:04.587359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7736,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94364104,"statfs":{"total":96636764160,"available":96628842496,"internally_reserved":0,"allocated":114688,"data_stored":31159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8700,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363140,"statfs":{"total":96636764160,"available":96627855360,"internally_reserved":0,"allocated":708608,"data_stored":621527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8708,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363132,"statfs":{"total":96636764160,"available":96627847168,"internally_reserved":0,"allocated":716800,"data_stored":624993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T18:25:09.588 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-29T18:25:09.588 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-29T18:25:09.588 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-29T18:25:09.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T18:25:09.931 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T18:25:09.932 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-29T18:25:09.945 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-29T18:25:09.945 INFO:teuthology.run_tasks:Running task exec... 2023-09-29T18:25:09.955 INFO:teuthology.task.exec:Executing custom commands... 2023-09-29T18:25:09.955 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T18:25:09.955 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2023-09-29T18:25:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cache' created 2023-09-29T18:25:10.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2023-09-29T18:25:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2023-09-29T18:25:11.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2023-09-29T18:25:12.597 INFO:teuthology.orchestra.run.smithi143.stderr:set cache-mode for pool 'cache' to writeback 2023-09-29T18:25:12.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2023-09-29T18:25:13.606 INFO:teuthology.orchestra.run.smithi143.stderr:overlay for 'rbd' is now (or already was) 'cache' 2023-09-29T18:25:13.618 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2023-09-29T18:25:14.608 INFO:teuthology.orchestra.run.smithi143.stderr:set pool 3 hit_set_type to bloom 2023-09-29T18:25:14.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2023-09-29T18:25:15.632 INFO:teuthology.orchestra.run.smithi143.stderr:set pool 3 hit_set_count to 8 2023-09-29T18:25:15.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2023-09-29T18:25:16.636 INFO:teuthology.orchestra.run.smithi143.stderr:set pool 3 hit_set_period to 60 2023-09-29T18:25:16.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2023-09-29T18:25:17.637 INFO:teuthology.orchestra.run.smithi143.stderr:set pool 3 target_max_objects to 250 2023-09-29T18:25:17.650 INFO:teuthology.run_tasks:Running task workunit... 2023-09-29T18:25:17.663 INFO:tasks.workunit:Pulling workunits from ref 2c186684db7f9ea0cdb098fc86b350a0fdd26eba 2023-09-29T18:25:17.663 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-09-29T18:25:17.663 DEBUG:teuthology.orchestra.run.smithi143:> stat -- /home/ubuntu/cephtest/mnt.0 2023-09-29T18:25:17.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T18:25:17.678 INFO:teuthology.orchestra.run.smithi143.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-09-29T18:25:17.679 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-09-29T18:25:17.734 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-09-29T18:25:17.734 DEBUG:teuthology.orchestra.run.smithi143:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-09-29T18:25:17.788 INFO:tasks.workunit:timeout=3h 2023-09-29T18:25:17.788 INFO:tasks.workunit:cleanup=True 2023-09-29T18:25:17.789 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2c186684db7f9ea0cdb098fc86b350a0fdd26eba 2023-09-29T18:25:17.843 INFO:tasks.workunit.client.0.smithi143.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-09-29T18:26:01.658 INFO:tasks.workunit.client.0.smithi143.stderr:Note: switching to '2c186684db7f9ea0cdb098fc86b350a0fdd26eba'. 2023-09-29T18:26:01.658 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.658 INFO:tasks.workunit.client.0.smithi143.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-09-29T18:26:01.658 INFO:tasks.workunit.client.0.smithi143.stderr:changes and commit them, and you can discard any commits you make in this 2023-09-29T18:26:01.658 INFO:tasks.workunit.client.0.smithi143.stderr:state without impacting any branches by switching back to a branch. 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr:If you want to create a new branch to retain commits you create, you may 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr:do so (now or later) by using -c with the switch command. Example: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: git switch -c 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr:Or undo this operation with: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: git switch - 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-09-29T18:26:01.659 INFO:tasks.workunit.client.0.smithi143.stderr: 2023-09-29T18:26:01.660 INFO:tasks.workunit.client.0.smithi143.stderr:HEAD is now at 2c186684db7 Merge branch 'wip-62711-quincy' of https://github.com/idryomov/ceph into wip-yuri3-testing-2023-09-27-0919-quincy 2023-09-29T18:26:01.663 DEBUG:teuthology.orchestra.run.smithi143:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-09-29T18:26:01.730 INFO:tasks.workunit.client.0.smithi143.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-09-29T18:26:01.730 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-29T18:26:01.730 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-09-29T18:26:02.271 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-09-29T18:26:02.333 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-09-29T18:26:02.369 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-29T18:26:02.370 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-29T18:26:02.371 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-09-29T18:26:02.406 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-29T18:26:02.409 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T18:26:02.409 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-09-29T18:26:02.464 INFO:tasks.workunit:Running workunits matching rbd/cli_generic.sh on client.0... 2023-09-29T18:26:02.466 INFO:tasks.workunit:Running workunit rbd/cli_generic.sh... 2023-09-29T18:26:02.466 DEBUG:teuthology.orchestra.run.smithi143:workunit test rbd/cli_generic.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c186684db7f9ea0cdb098fc86b350a0fdd26eba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-09-29T18:26:02.523 INFO:tasks.workunit.client.0.smithi143.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-09-29T18:26:02.524 INFO:tasks.workunit.client.0.smithi143.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2023-09-29T18:26:02.524 INFO:tasks.workunit.client.0.smithi143.stderr:++ TIMEOUT=300 2023-09-29T18:26:02.524 INFO:tasks.workunit.client.0.smithi143.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2023-09-29T18:26:02.524 INFO:tasks.workunit.client.0.smithi143.stderr:++ MAX_TIMEOUT=15 2023-09-29T18:26:02.524 INFO:tasks.workunit.client.0.smithi143.stderr:++ PG_NUM=4 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:++ TMPDIR=/tmp 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:++ TESTDIR=/home/ubuntu/cephtest 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:++ type xmlstarlet 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:++ XMLSTARLET=xmlstarlet 2023-09-29T18:26:02.525 INFO:tasks.workunit.client.0.smithi143.stderr:+++ uname 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:++ '[' Linux = FreeBSD ']' 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:++ SED=sed 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:++ AWK=awk 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:+++ stty -a 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:+++ head -1 2023-09-29T18:26:02.526 INFO:tasks.workunit.client.0.smithi143.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:++ termwidth= 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:++ DIFFCOLOPTS='-y ' 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:++ KERNCORE=kernel.core_pattern 2023-09-29T18:26:02.527 INFO:tasks.workunit.client.0.smithi143.stderr:++ EXTRA_OPTS= 2023-09-29T18:26:02.531 INFO:tasks.workunit.client.0.smithi143.stderr:++ test '' = TESTS 2023-09-29T18:26:02.531 INFO:tasks.workunit.client.0.smithi143.stderr:+ export RBD_FORCE_ALLOW_V1=1 2023-09-29T18:26:02.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ RBD_FORCE_ALLOW_V1=1 2023-09-29T18:26:02.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:02.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:02.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v '^0$' 2023-09-29T18:26:02.582 INFO:tasks.workunit.client.0.smithi143.stderr:+ IMGS='testimg1 testimg2 testimg3 testimg4 testimg5 testimg6 testimg-diff1 testimg-diff2 testimg-diff3 foo foo2 bar bar2 test1 test2 test3 test4 clone2' 2023-09-29T18:26:02.582 INFO:tasks.workunit.client.0.smithi143.stderr:+ tiered=0 2023-09-29T18:26:02.582 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd dump 2023-09-29T18:26:02.583 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^pool' 2023-09-29T18:26:02.583 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ''\''rbd'\''' 2023-09-29T18:26:02.583 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep tier 2023-09-29T18:26:03.002 INFO:tasks.workunit.client.0.smithi143.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 lfor 18/18/18 flags hashpspool tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2023-09-29T18:26:03.002 INFO:tasks.workunit.client.0.smithi143.stderr:+ tiered=1 2023-09-29T18:26:03.008 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_pool_image_args 2023-09-29T18:26:03.008 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing pool and image args...' 2023-09-29T18:26:03.008 INFO:tasks.workunit.client.0.smithi143.stdout:testing pool and image args... 2023-09-29T18:26:03.009 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:03.009 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.083 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.152 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.222 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.292 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.362 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.437 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.515 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.670 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.746 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.815 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:03.953 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:04.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:04.097 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:04.176 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:04.254 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:04.331 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-09-29T18:26:04.749 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'test' does not exist 2023-09-29T18:26:04.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool create test 32 2023-09-29T18:26:05.799 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'test' already exists 2023-09-29T18:26:05.808 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd pool init test 2023-09-29T18:26:08.691 INFO:tasks.workunit.client.0.smithi143.stderr:+ truncate -s 1 /tmp/empty /tmp/empty@snap 2023-09-29T18:26:08.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:08.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:08.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:26:08.741 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:08.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 1 test1 2023-09-29T18:26:08.770 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:08.770+0000 7f6770329500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:10.696 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:10.697 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test1 2023-09-29T18:26:10.726 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import --image test2 /tmp/empty 2023-09-29T18:26:10.756 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:10.756+0000 7f8ac2cf0500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:10.765 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:10.765 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:10.768 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:10.768 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test2 2023-09-29T18:26:10.795 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --dest test3 import /tmp/empty 2023-09-29T18:26:10.823 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:10.824+0000 7fc9efab4500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:10.833 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:10.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:10.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test3 2023-09-29T18:26:10.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import /tmp/empty foo 2023-09-29T18:26:10.889 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:10.890+0000 7fc5596be500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:10.904 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:10.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:10.907 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q foo 2023-09-29T18:26:10.933 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import --dest test/empty@snap /tmp/empty 2023-09-29T18:26:10.954 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-09-29T18:26:10.955 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:26:10.955 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import /tmp/empty test/empty@snap 2023-09-29T18:26:10.975 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-09-29T18:26:10.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:26:10.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import --image test/empty@snap /tmp/empty 2023-09-29T18:26:10.998 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-09-29T18:26:10.998 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:10.999 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:26:11.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import /tmp/empty@snap 2023-09-29T18:26:11.019 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-09-29T18:26:11.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:26:11.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:11.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:11.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:26:11.047 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:11.047 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import /tmp/empty test/test1 2023-09-29T18:26:11.073 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:11.073+0000 7f7bbf9fb500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:12.713 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:12.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:12.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test1 2023-09-29T18:26:12.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p test import /tmp/empty test2 2023-09-29T18:26:12.776 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:12.777+0000 7f2de8a6f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:12.786 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:12.786 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-09-29T18:26:12.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:12.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test2 2023-09-29T18:26:12.817 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --image test3 -p test import /tmp/empty 2023-09-29T18:26:12.846 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:12.847+0000 7f90d1935500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:12.857 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:12.857 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:12.858 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-09-29T18:26:12.860 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:12.860 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test3 2023-09-29T18:26:12.887 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --image test4 -p test import /tmp/empty 2023-09-29T18:26:12.916 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:12.917+0000 7f5858bce500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:12.926 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:12.927 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:12.927 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-09-29T18:26:12.929 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:12.929 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test4 2023-09-29T18:26:12.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --dest test5 -p test import /tmp/empty 2023-09-29T18:26:12.986 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:12.986+0000 7fbabcdd3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:12.997 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:12.997 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-09-29T18:26:13.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test5 2023-09-29T18:26:13.027 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --dest test6 --dest-pool test import /tmp/empty 2023-09-29T18:26:13.056 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:13.056+0000 7f95da106500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:13.069 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:13.072 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.072 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test6 2023-09-29T18:26:13.099 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --image test7 --dest-pool test import /tmp/empty 2023-09-29T18:26:13.129 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:13.129+0000 7fcdb45ac500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:13.139 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:13.140 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:13.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test7 2023-09-29T18:26:13.170 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --image test/test8 import /tmp/empty 2023-09-29T18:26:13.197 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:13.197+0000 7fcbdd40a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:13.209 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:13.209 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: --image is deprecated, use --dest 2023-09-29T18:26:13.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test8 2023-09-29T18:26:13.238 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --dest test/test9 import /tmp/empty 2023-09-29T18:26:13.265 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:13.266+0000 7fedcb476500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:13.276 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:13.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test9 2023-09-29T18:26:13.307 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import --pool test /tmp/empty 2023-09-29T18:26:13.334 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:13.334+0000 7fc838c16500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:13.345 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:26:13.345 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-09-29T18:26:13.348 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:13.348 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q empty 2023-09-29T18:26:13.375 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy test/test9 test10 2023-09-29T18:26:14.718 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:26:14.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:14.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -qv test10 2023-09-29T18:26:14.752 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:14.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test10 2023-09-29T18:26:14.781 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy test/test9 test/test10 2023-09-29T18:26:14.828 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:26:14.831 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:14.831 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test10 2023-09-29T18:26:14.859 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy --pool test test10 --dest-pool test test11 2023-09-29T18:26:14.911 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:26:14.914 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:14.914 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -q test11 2023-09-29T18:26:14.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy --dest-pool rbd --pool test test11 test12 2023-09-29T18:26:14.996 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:26:14.999 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:15.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test12 2023-09-29T18:26:15.026 INFO:tasks.workunit.client.0.smithi143.stdout:test12 2023-09-29T18:26:15.026 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls test 2023-09-29T18:26:15.026 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -qv test12 2023-09-29T18:26:15.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -f /tmp/empty /tmp/empty@snap 2023-09-29T18:26:15.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-09-29T18:26:15.826 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'test' does not exist 2023-09-29T18:26:15.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:15.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm foo 2023-09-29T18:26:15.895 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:15.897 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:15.897 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:26:15.936 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:15.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:15.939 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test10 2023-09-29T18:26:16.020 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:16.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:16.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test12 2023-09-29T18:26:16.096 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:16.098 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:16.098 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:26:16.134 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:16.136 INFO:tasks.workunit.client.0.smithi143.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-09-29T18:26:16.136 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test3 2023-09-29T18:26:16.172 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:16.174 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_rename 2023-09-29T18:26:16.174 INFO:tasks.workunit.client.0.smithi143.stdout:testing rename... 2023-09-29T18:26:16.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing rename...' 2023-09-29T18:26:16.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:16.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.243 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.311 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.380 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.448 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.517 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.586 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.663 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.736 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.811 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.886 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:16.954 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.025 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.095 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.169 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.241 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.318 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:17.460 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 foo 2023-09-29T18:26:17.482 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:17.488 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:17.489+0000 7f07ff643500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:17.494 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 bar 2023-09-29T18:26:17.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename foo foo2 2023-09-29T18:26:17.584 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename foo2 bar 2023-09-29T18:26:17.584 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep exists 2023-09-29T18:26:17.625 INFO:tasks.workunit.client.0.smithi143.stdout:2023-09-29T18:26:17.617+0000 7ffb82e3d500 -1 librbd::Operations: rbd image bar already exists 2023-09-29T18:26:17.625 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: rename error: (17) File exists 2023-09-29T18:26:17.625 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename bar bar2 2023-09-29T18:26:17.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename bar2 foo2 2023-09-29T18:26:17.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep exists 2023-09-29T18:26:17.711 INFO:tasks.workunit.client.0.smithi143.stdout:2023-09-29T18:26:17.708+0000 7f1afdba7500 -1 librbd::Operations: rbd image foo2 already exists 2023-09-29T18:26:17.711 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: rename error: (17) File exists 2023-09-29T18:26:17.711 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool create rbd2 8 2023-09-29T18:26:18.826 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' already exists 2023-09-29T18:26:18.835 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd pool init rbd2 2023-09-29T18:26:21.720 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -p rbd2 -s 1 foo 2023-09-29T18:26:21.752 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:21.752+0000 7f29a6316500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:23.734 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename rbd2/foo rbd2/bar 2023-09-29T18:26:23.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls 2023-09-29T18:26:23.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep bar 2023-09-29T18:26:23.814 INFO:tasks.workunit.client.0.smithi143.stdout:bar 2023-09-29T18:26:23.814 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename rbd2/bar foo 2023-09-29T18:26:24.039 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:24.039+0000 7fc8d4bfc700 0 --2- 172.21.15.143:0/171617355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f8c6256150 0x55f8c62f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:26:24.060 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename --pool rbd2 foo bar 2023-09-29T18:26:24.109 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename rbd2/bar --dest-pool rbd foo 2023-09-29T18:26:24.132 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: mv/rename across pools not supported 2023-09-29T18:26:24.132 INFO:tasks.workunit.client.0.smithi143.stderr:source pool: rbd2 dest pool: rbd 2023-09-29T18:26:24.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rename --pool rbd2 bar --dest-pool rbd2 foo 2023-09-29T18:26:24.359 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:24.359+0000 7f8021a2f700 0 --2- 172.21.15.143:0/242849489 >> v2:172.21.15.143:3300/0 conn(0x56020c0d36d0 0x56020c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:26:24.386 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls 2023-09-29T18:26:24.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep foo 2023-09-29T18:26:24.414 INFO:tasks.workunit.client.0.smithi143.stdout:foo 2023-09-29T18:26:24.414 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-09-29T18:26:24.845 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' does not exist 2023-09-29T18:26:24.854 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:24.854 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:24.928 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.003 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.154 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.228 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.304 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.377 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.446 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.515 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.584 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.665 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.743 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.879 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:25.947 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.016 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.085 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.154 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_ls 2023-09-29T18:26:26.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing ls...' 2023-09-29T18:26:26.227 INFO:tasks.workunit.client.0.smithi143.stdout:testing ls... 2023-09-29T18:26:26.228 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:26.228 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.304 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.382 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.455 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.526 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.600 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.668 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.737 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.807 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:26.944 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.013 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.082 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.150 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.219 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.289 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.368 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.436 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:27.505 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 test1 2023-09-29T18:26:27.528 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:27.533 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:27.533+0000 7f6c279b3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:27.539 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 test2 2023-09-29T18:26:27.562 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:27.568 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:27.568+0000 7f43503f5500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:27.578 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.578 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:26:27.607 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:26:27.607 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.607 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:26:27.633 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:26:27.634 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.634 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:27.634 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:26:27.660 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:26:27.661 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:27.661 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*1 MiB.*1' 2023-09-29T18:26:27.693 INFO:tasks.workunit.client.0.smithi143.stdout:test1 1 MiB 1 2023-09-29T18:26:27.694 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:27.694 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*1 MiB.*1' 2023-09-29T18:26:27.727 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 1 2023-09-29T18:26:27.727 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:26:27.766 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:27.768 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:26:27.803 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:27.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:26:27.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test2 2023-09-29T18:26:27.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:26:27.922 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:26:27.923 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.923 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:26:27.949 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:26:27.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:27.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:27.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:26:27.976 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:26:27.976 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:27.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*1 MiB.*2' 2023-09-29T18:26:28.012 INFO:tasks.workunit.client.0.smithi143.stdout:test1 1 MiB 2 2023-09-29T18:26:28.012 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:28.012 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*1 MiB.*2' 2023-09-29T18:26:28.048 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 2 2023-09-29T18:26:28.048 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:26:28.128 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:28.127+0000 7fa0ae7bb700 0 -- 172.21.15.143:0/1536777658 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fa094008740 msgr2=0x7fa09400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:28.128 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:28.131 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:26:28.228 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:28.228+0000 7f1c01894700 0 -- 172.21.15.143:0/4253133863 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f1be8008740 msgr2=0x7f1be800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:28.229 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:28.232 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:26:28.273 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 test2 2023-09-29T18:26:28.297 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:28.302 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:28.302+0000 7fac7edf2500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:28.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:28.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:26:28.336 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:26:28.336 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:28.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:26:28.363 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:26:28.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:28.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:28.364 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:26:28.390 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:26:28.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:28.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*1 MiB.*2' 2023-09-29T18:26:28.426 INFO:tasks.workunit.client.0.smithi143.stdout:test1 1 MiB 2 2023-09-29T18:26:28.426 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:28.426 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*1 MiB.*1' 2023-09-29T18:26:28.461 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 1 2023-09-29T18:26:28.462 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:28.462 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.613 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.689 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.828 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:28.965 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.103 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.173 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.251 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.327 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.404 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.550 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.622 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.691 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.760 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:29.829 INFO:tasks.workunit.client.0.smithi143.stderr:++ seq -w 00 99 2023-09-29T18:26:29.841 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:29.842 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.00 -s 1 2023-09-29T18:26:29.869 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:29.869+0000 7f44e5b9d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:29.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:29.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.01 -s 1 2023-09-29T18:26:29.903 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:29.902+0000 7fb4c5246500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:29.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:29.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.02 -s 1 2023-09-29T18:26:29.936 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:29.935+0000 7f39d6511500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:29.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:29.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.03 -s 1 2023-09-29T18:26:29.970 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:29.969+0000 7f6530f20500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:29.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:29.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.04 -s 1 2023-09-29T18:26:30.005 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.004+0000 7fbfd33e5500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.015 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.016 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.05 -s 1 2023-09-29T18:26:30.043 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.043+0000 7f18f618a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.06 -s 1 2023-09-29T18:26:30.082 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.082+0000 7ff5015a7500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.07 -s 1 2023-09-29T18:26:30.116 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.116+0000 7f07cfa89500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.124 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.124 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.08 -s 1 2023-09-29T18:26:30.152 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.151+0000 7f0932914500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.163 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.163 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.09 -s 1 2023-09-29T18:26:30.191 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.190+0000 7ff97d8a7500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.197 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.197 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.10 -s 1 2023-09-29T18:26:30.225 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.224+0000 7f550a1f4500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.230 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.230 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.11 -s 1 2023-09-29T18:26:30.258 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.258+0000 7fe0f4f62500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.266 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.266 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.12 -s 1 2023-09-29T18:26:30.293 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.293+0000 7f8837fb6500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.13 -s 1 2023-09-29T18:26:30.330 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.330+0000 7f592aca0500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.338 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.338 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.14 -s 1 2023-09-29T18:26:30.365 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.365+0000 7f9830fb7500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.371 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.371 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.15 -s 1 2023-09-29T18:26:30.399 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.399+0000 7f36c52d1500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.410 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.410 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.16 -s 1 2023-09-29T18:26:30.437 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.437+0000 7efd1995a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.448 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.448 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.17 -s 1 2023-09-29T18:26:30.476 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.476+0000 7f766222a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.485 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.485 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.18 -s 1 2023-09-29T18:26:30.513 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.513+0000 7f3cc0113500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.525 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.525 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.19 -s 1 2023-09-29T18:26:30.553 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.552+0000 7fec1815d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.563 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.564 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.20 -s 1 2023-09-29T18:26:30.591 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.591+0000 7f35b7bc3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.596 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.596 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.21 -s 1 2023-09-29T18:26:30.624 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.624+0000 7fe66e6c1500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.22 -s 1 2023-09-29T18:26:30.658 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.657+0000 7f866f658500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.663 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.663 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.23 -s 1 2023-09-29T18:26:30.691 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.691+0000 7ff6ad02b500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.697 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.697 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.24 -s 1 2023-09-29T18:26:30.725 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.724+0000 7fa81c4a8500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.730 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.730 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.25 -s 1 2023-09-29T18:26:30.758 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.757+0000 7f7892cd4500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.26 -s 1 2023-09-29T18:26:30.796 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.796+0000 7fdecfeef500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.27 -s 1 2023-09-29T18:26:30.833 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.833+0000 7f3318967500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.841 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.28 -s 1 2023-09-29T18:26:30.869 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.868+0000 7f35e75a3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.29 -s 1 2023-09-29T18:26:30.902 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.902+0000 7fbd33aff500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.907 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.30 -s 1 2023-09-29T18:26:30.935 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.935+0000 7efc92085500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.31 -s 1 2023-09-29T18:26:30.974 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:30.974+0000 7f0669d28500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:30.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:30.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.32 -s 1 2023-09-29T18:26:31.012 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.012+0000 7f2283b28500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.021 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.021 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.33 -s 1 2023-09-29T18:26:31.049 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.049+0000 7f977c1de500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.060 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.061 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.34 -s 1 2023-09-29T18:26:31.088 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.088+0000 7f7efc52d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.094 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.094 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.35 -s 1 2023-09-29T18:26:31.122 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.122+0000 7f0e22e05500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.127 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.128 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.36 -s 1 2023-09-29T18:26:31.155 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.155+0000 7fec2edda500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.167 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.167 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.37 -s 1 2023-09-29T18:26:31.195 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.194+0000 7f5da3565500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.204 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.204 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.38 -s 1 2023-09-29T18:26:31.232 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.231+0000 7f129ce98500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.39 -s 1 2023-09-29T18:26:31.266 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.265+0000 7f7c2387f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.271 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.271 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.40 -s 1 2023-09-29T18:26:31.299 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.299+0000 7f4ce7963500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.305 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.305 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.41 -s 1 2023-09-29T18:26:31.333 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.333+0000 7f28abbdb500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.340 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.340 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.42 -s 1 2023-09-29T18:26:31.368 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.367+0000 7f47827b0500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.373 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.373 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.43 -s 1 2023-09-29T18:26:31.401 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.401+0000 7f8215f4f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.406 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.407 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.44 -s 1 2023-09-29T18:26:31.435 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.434+0000 7fe2ffa45500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.45 -s 1 2023-09-29T18:26:31.468 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.468+0000 7f0647738500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.475 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.475 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.46 -s 1 2023-09-29T18:26:31.502 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.502+0000 7fb9d3c1d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.508 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.508 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.47 -s 1 2023-09-29T18:26:31.537 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.536+0000 7efc63b06500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.542 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.542 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.48 -s 1 2023-09-29T18:26:31.570 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.570+0000 7f1f5593e500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.580 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.580 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.49 -s 1 2023-09-29T18:26:31.608 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.607+0000 7f6635eaa500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.615 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.50 -s 1 2023-09-29T18:26:31.642 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.642+0000 7ff100128500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.648 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.648 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.51 -s 1 2023-09-29T18:26:31.676 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.676+0000 7fef2ccbf500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.683 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.683 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.52 -s 1 2023-09-29T18:26:31.711 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.710+0000 7f24c2185500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.53 -s 1 2023-09-29T18:26:31.745 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.745+0000 7f90a8453500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.751 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.751 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.54 -s 1 2023-09-29T18:26:31.779 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.779+0000 7f15bef84500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.55 -s 1 2023-09-29T18:26:31.818 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.818+0000 7f4cd70ae500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.836 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.56 -s 1 2023-09-29T18:26:31.864 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.864+0000 7f1e78ef9500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.877 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.877 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.57 -s 1 2023-09-29T18:26:31.905 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.904+0000 7f58c0391500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.915 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.915 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.58 -s 1 2023-09-29T18:26:31.943 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.942+0000 7fe1e6831500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.952 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.952 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.59 -s 1 2023-09-29T18:26:31.980 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:31.979+0000 7fdf467ed500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:31.987 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:31.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.60 -s 1 2023-09-29T18:26:32.015 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.015+0000 7f315a8df500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.61 -s 1 2023-09-29T18:26:32.049 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.049+0000 7ff90e89e500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.056 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.056 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.62 -s 1 2023-09-29T18:26:32.085 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.084+0000 7f6485849500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.63 -s 1 2023-09-29T18:26:32.128 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.127+0000 7f244709f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.137 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.137 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.64 -s 1 2023-09-29T18:26:32.165 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.165+0000 7fe248d04500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.171 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.171 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.65 -s 1 2023-09-29T18:26:32.199 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.198+0000 7f997a7f3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.206 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.206 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.66 -s 1 2023-09-29T18:26:32.233 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.233+0000 7f3f3807c500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.67 -s 1 2023-09-29T18:26:32.267 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.267+0000 7ff3aa3e9500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.275 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.275 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.68 -s 1 2023-09-29T18:26:32.303 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.302+0000 7f8e4e83f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.69 -s 1 2023-09-29T18:26:32.337 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.336+0000 7f8822ec5500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.345 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.345 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.70 -s 1 2023-09-29T18:26:32.373 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.373+0000 7ff496eb4500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.382 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.383 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.71 -s 1 2023-09-29T18:26:32.410 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.410+0000 7f9d306f6500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.421 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.421 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.72 -s 1 2023-09-29T18:26:32.449 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.449+0000 7f4459c87500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.460 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.460 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.73 -s 1 2023-09-29T18:26:32.487 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.487+0000 7f7166cd2500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.495 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.495 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.74 -s 1 2023-09-29T18:26:32.523 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.523+0000 7fc1719e2500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.75 -s 1 2023-09-29T18:26:32.560 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.560+0000 7fa816e75500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.76 -s 1 2023-09-29T18:26:32.597 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.596+0000 7f24762d3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.607 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.607 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.77 -s 1 2023-09-29T18:26:32.634 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.634+0000 7f8e94734500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.643 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.644 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.78 -s 1 2023-09-29T18:26:32.671 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.671+0000 7f97dbe7a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.677 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.677 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.79 -s 1 2023-09-29T18:26:32.706 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.705+0000 7f279734e500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.711 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.711 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.80 -s 1 2023-09-29T18:26:32.740 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.739+0000 7f35cb3b2500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.755 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.755 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.81 -s 1 2023-09-29T18:26:32.782 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.781+0000 7f20c2e93500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.82 -s 1 2023-09-29T18:26:32.817 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.817+0000 7f5d8f9dc500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.83 -s 1 2023-09-29T18:26:32.853 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.852+0000 7f9b22f0d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.858 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.859 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.84 -s 1 2023-09-29T18:26:32.886 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.886+0000 7efe13ac7500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.85 -s 1 2023-09-29T18:26:32.921 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.921+0000 7f50de5e9500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.931 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.86 -s 1 2023-09-29T18:26:32.959 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.958+0000 7f603c08e500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:32.966 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:32.966 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.87 -s 1 2023-09-29T18:26:32.994 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:32.993+0000 7fd8b669a500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.88 -s 1 2023-09-29T18:26:33.028 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.028+0000 7f87bdfe0500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.89 -s 1 2023-09-29T18:26:33.064 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.064+0000 7f7d69b63500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.070 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.071 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.90 -s 1 2023-09-29T18:26:33.100 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.099+0000 7f47bb12c500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.114 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.114 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.91 -s 1 2023-09-29T18:26:33.141 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.140+0000 7fbb65b62500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.146 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.146 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.92 -s 1 2023-09-29T18:26:33.174 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.174+0000 7ffa3e033500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.93 -s 1 2023-09-29T18:26:33.208 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.208+0000 7fd2b83f3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.216 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.216 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.94 -s 1 2023-09-29T18:26:33.244 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.244+0000 7f3f89dc3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.253 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.253 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.95 -s 1 2023-09-29T18:26:33.281 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.280+0000 7fd6fdcbb500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.289 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.289 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.96 -s 1 2023-09-29T18:26:33.317 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.317+0000 7f8065b40500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.326 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.327 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.97 -s 1 2023-09-29T18:26:33.354 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.354+0000 7f2a058bf500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.367 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.367 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.98 -s 1 2023-09-29T18:26:33.395 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.394+0000 7f40f38be500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.402 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.403 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.99 -s 1 2023-09-29T18:26:33.430 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.430+0000 7f793b1fd500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:33.439 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:33.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:33.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 100 2023-09-29T18:26:33.467 INFO:tasks.workunit.client.0.smithi143.stdout:100 2023-09-29T18:26:33.468 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:33.468 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep image 2023-09-29T18:26:33.468 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:33.468 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 100 2023-09-29T18:26:33.534 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.533+0000 7f060e818700 0 -- 172.21.15.143:0/1010703976 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55af5ce13d40 msgr2=0x55af5ce1a0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:33.543 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:33.543+0000 7f060e818700 0 -- 172.21.15.143:0/1010703976 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f05f003a370 msgr2=0x7f05f003c770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:33.712 INFO:tasks.workunit.client.0.smithi143.stdout:100 2023-09-29T18:26:33.712 INFO:tasks.workunit.client.0.smithi143.stderr:++ seq -w 00 99 2023-09-29T18:26:33.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.00 2023-09-29T18:26:33.752 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:33.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:33.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.01 2023-09-29T18:26:33.991 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.02 2023-09-29T18:26:34.029 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.031 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.03 2023-09-29T18:26:34.067 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.070 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.070 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.04 2023-09-29T18:26:34.108 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.111 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.111 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.05 2023-09-29T18:26:34.150 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.151 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.151 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.06 2023-09-29T18:26:34.188 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.07 2023-09-29T18:26:34.228 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.230 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.231 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.08 2023-09-29T18:26:34.271 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.273 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.274 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.09 2023-09-29T18:26:34.309 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.312 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.312 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.10 2023-09-29T18:26:34.349 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.351 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.351 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.11 2023-09-29T18:26:34.387 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.12 2023-09-29T18:26:34.426 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.428 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.428 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.13 2023-09-29T18:26:34.470 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.472 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.473 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.14 2023-09-29T18:26:34.508 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.510 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.510 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.15 2023-09-29T18:26:34.549 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.551 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.551 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.16 2023-09-29T18:26:34.590 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.592 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.592 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.17 2023-09-29T18:26:34.628 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.631 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.631 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.18 2023-09-29T18:26:34.667 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.669 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.670 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.19 2023-09-29T18:26:34.708 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.20 2023-09-29T18:26:34.749 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.751 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.751 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.21 2023-09-29T18:26:34.790 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.22 2023-09-29T18:26:34.837 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.839 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.839 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.23 2023-09-29T18:26:34.876 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.24 2023-09-29T18:26:34.914 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.916 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.917 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.25 2023-09-29T18:26:34.954 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.26 2023-09-29T18:26:34.993 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:34.994 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:34.994 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.27 2023-09-29T18:26:35.035 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.28 2023-09-29T18:26:35.076 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.078 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.29 2023-09-29T18:26:35.115 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.118 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.118 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.30 2023-09-29T18:26:35.164 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.166 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.166 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.31 2023-09-29T18:26:35.205 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.207 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.207 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.32 2023-09-29T18:26:35.243 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.245 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.245 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.33 2023-09-29T18:26:35.282 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.284 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.284 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.34 2023-09-29T18:26:35.321 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.323 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.323 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.35 2023-09-29T18:26:35.359 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.361 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.361 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.36 2023-09-29T18:26:35.397 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.400 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.400 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.37 2023-09-29T18:26:35.439 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.442 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.442 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.38 2023-09-29T18:26:35.483 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.39 2023-09-29T18:26:35.526 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.528 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.529 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.40 2023-09-29T18:26:35.570 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.572 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.572 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.41 2023-09-29T18:26:35.612 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.42 2023-09-29T18:26:35.655 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.657 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.657 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.43 2023-09-29T18:26:35.697 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.44 2023-09-29T18:26:35.739 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.45 2023-09-29T18:26:35.780 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.783 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.783 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.46 2023-09-29T18:26:35.821 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.824 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.824 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.47 2023-09-29T18:26:35.861 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:35.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:35.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.48 2023-09-29T18:26:36.102 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.49 2023-09-29T18:26:36.140 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.143 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.143 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.50 2023-09-29T18:26:36.179 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.181 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.181 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.51 2023-09-29T18:26:36.217 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.219 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.220 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.52 2023-09-29T18:26:36.257 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.259 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.260 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.53 2023-09-29T18:26:36.295 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.298 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.298 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.54 2023-09-29T18:26:36.337 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.340 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.340 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.55 2023-09-29T18:26:36.380 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.383 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.383 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.56 2023-09-29T18:26:36.424 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.426 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.426 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.57 2023-09-29T18:26:36.466 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.58 2023-09-29T18:26:36.508 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.510 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.510 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.59 2023-09-29T18:26:36.552 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.554 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.554 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.60 2023-09-29T18:26:36.601 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.604 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.604 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.61 2023-09-29T18:26:36.646 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.649 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.649 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.62 2023-09-29T18:26:36.688 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.690 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.690 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.63 2023-09-29T18:26:36.728 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.730 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.730 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.64 2023-09-29T18:26:36.766 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.65 2023-09-29T18:26:36.809 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.812 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.812 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.66 2023-09-29T18:26:36.850 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.67 2023-09-29T18:26:36.889 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.68 2023-09-29T18:26:36.928 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.931 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.931 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.69 2023-09-29T18:26:36.967 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:36.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:36.970 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.70 2023-09-29T18:26:37.008 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.010 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.010 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.71 2023-09-29T18:26:37.049 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.052 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.052 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.72 2023-09-29T18:26:37.089 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.091 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.73 2023-09-29T18:26:37.139 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.74 2023-09-29T18:26:37.178 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.75 2023-09-29T18:26:37.219 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.222 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.222 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.76 2023-09-29T18:26:37.458 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.460 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.461 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.77 2023-09-29T18:26:37.505 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.507 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.507 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.78 2023-09-29T18:26:37.546 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.549 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.549 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.79 2023-09-29T18:26:37.590 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.80 2023-09-29T18:26:37.634 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.636 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.636 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.81 2023-09-29T18:26:37.677 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.679 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.679 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.82 2023-09-29T18:26:37.722 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.724 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.724 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.83 2023-09-29T18:26:37.761 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.84 2023-09-29T18:26:37.804 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.807 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.85 2023-09-29T18:26:37.851 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.86 2023-09-29T18:26:37.896 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.899 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.87 2023-09-29T18:26:37.936 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.939 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.88 2023-09-29T18:26:37.983 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:37.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:37.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.89 2023-09-29T18:26:38.029 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.90 2023-09-29T18:26:38.073 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.075 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.075 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.91 2023-09-29T18:26:38.114 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.92 2023-09-29T18:26:38.163 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.165 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.165 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.93 2023-09-29T18:26:38.206 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.208 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.208 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.94 2023-09-29T18:26:38.244 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.247 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.247 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.95 2023-09-29T18:26:38.289 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.291 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.292 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.96 2023-09-29T18:26:38.534 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.97 2023-09-29T18:26:38.580 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.582 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.582 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.98 2023-09-29T18:26:38.622 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.625 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.625 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.99 2023-09-29T18:26:38.662 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:38.664 INFO:tasks.workunit.client.0.smithi143.stderr:++ seq -w 00 99 2023-09-29T18:26:38.665 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.665 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.00 --image-format 2 -s 1 2023-09-29T18:26:38.708 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.708 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.01 --image-format 2 -s 1 2023-09-29T18:26:38.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.02 --image-format 2 -s 1 2023-09-29T18:26:38.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.03 --image-format 2 -s 1 2023-09-29T18:26:38.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.04 --image-format 2 -s 1 2023-09-29T18:26:38.883 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.883 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.05 --image-format 2 -s 1 2023-09-29T18:26:38.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.06 --image-format 2 -s 1 2023-09-29T18:26:38.976 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:38.976 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.07 --image-format 2 -s 1 2023-09-29T18:26:39.018 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.018 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.08 --image-format 2 -s 1 2023-09-29T18:26:39.060 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.060 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.09 --image-format 2 -s 1 2023-09-29T18:26:39.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.10 --image-format 2 -s 1 2023-09-29T18:26:39.153 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.153 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.11 --image-format 2 -s 1 2023-09-29T18:26:39.199 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.199 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.12 --image-format 2 -s 1 2023-09-29T18:26:39.241 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.241 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.13 --image-format 2 -s 1 2023-09-29T18:26:39.288 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.288 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.14 --image-format 2 -s 1 2023-09-29T18:26:39.334 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.334 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.15 --image-format 2 -s 1 2023-09-29T18:26:39.386 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.16 --image-format 2 -s 1 2023-09-29T18:26:39.439 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.439 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.17 --image-format 2 -s 1 2023-09-29T18:26:39.485 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.485 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.18 --image-format 2 -s 1 2023-09-29T18:26:39.527 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.527 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.19 --image-format 2 -s 1 2023-09-29T18:26:39.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.20 --image-format 2 -s 1 2023-09-29T18:26:39.612 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.612 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.21 --image-format 2 -s 1 2023-09-29T18:26:39.656 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.656 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.22 --image-format 2 -s 1 2023-09-29T18:26:39.701 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.701 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.23 --image-format 2 -s 1 2023-09-29T18:26:39.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.24 --image-format 2 -s 1 2023-09-29T18:26:39.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.25 --image-format 2 -s 1 2023-09-29T18:26:39.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.841 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.26 --image-format 2 -s 1 2023-09-29T18:26:39.883 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.883 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.27 --image-format 2 -s 1 2023-09-29T18:26:39.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.28 --image-format 2 -s 1 2023-09-29T18:26:39.979 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:39.979 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.29 --image-format 2 -s 1 2023-09-29T18:26:40.027 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.027 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.30 --image-format 2 -s 1 2023-09-29T18:26:40.077 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.078 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.31 --image-format 2 -s 1 2023-09-29T18:26:40.119 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.119 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.32 --image-format 2 -s 1 2023-09-29T18:26:40.167 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.168 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.33 --image-format 2 -s 1 2023-09-29T18:26:40.209 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.209 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.34 --image-format 2 -s 1 2023-09-29T18:26:40.252 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.252 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.35 --image-format 2 -s 1 2023-09-29T18:26:40.300 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.300 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.36 --image-format 2 -s 1 2023-09-29T18:26:40.349 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.349 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.37 --image-format 2 -s 1 2023-09-29T18:26:40.395 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.395 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.38 --image-format 2 -s 1 2023-09-29T18:26:40.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.39 --image-format 2 -s 1 2023-09-29T18:26:40.489 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.489 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.40 --image-format 2 -s 1 2023-09-29T18:26:40.545 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.546 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.41 --image-format 2 -s 1 2023-09-29T18:26:40.590 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.591 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.42 --image-format 2 -s 1 2023-09-29T18:26:40.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.43 --image-format 2 -s 1 2023-09-29T18:26:40.691 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.691 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.44 --image-format 2 -s 1 2023-09-29T18:26:40.733 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.733 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.45 --image-format 2 -s 1 2023-09-29T18:26:40.777 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.777 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.46 --image-format 2 -s 1 2023-09-29T18:26:40.818 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.818 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.47 --image-format 2 -s 1 2023-09-29T18:26:40.859 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.859 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.48 --image-format 2 -s 1 2023-09-29T18:26:40.905 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.905 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.49 --image-format 2 -s 1 2023-09-29T18:26:40.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.50 --image-format 2 -s 1 2023-09-29T18:26:40.995 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:40.995 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.51 --image-format 2 -s 1 2023-09-29T18:26:41.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.52 --image-format 2 -s 1 2023-09-29T18:26:41.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.53 --image-format 2 -s 1 2023-09-29T18:26:41.130 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.130 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.54 --image-format 2 -s 1 2023-09-29T18:26:41.174 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.55 --image-format 2 -s 1 2023-09-29T18:26:41.221 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.221 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.56 --image-format 2 -s 1 2023-09-29T18:26:41.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.57 --image-format 2 -s 1 2023-09-29T18:26:41.310 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.311 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.58 --image-format 2 -s 1 2023-09-29T18:26:41.355 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.356 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.59 --image-format 2 -s 1 2023-09-29T18:26:41.398 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.398 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.60 --image-format 2 -s 1 2023-09-29T18:26:41.441 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.442 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.61 --image-format 2 -s 1 2023-09-29T18:26:41.484 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.484 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.62 --image-format 2 -s 1 2023-09-29T18:26:41.530 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.530 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.63 --image-format 2 -s 1 2023-09-29T18:26:41.579 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.579 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.64 --image-format 2 -s 1 2023-09-29T18:26:41.623 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.624 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.65 --image-format 2 -s 1 2023-09-29T18:26:41.667 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.667 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.66 --image-format 2 -s 1 2023-09-29T18:26:41.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.67 --image-format 2 -s 1 2023-09-29T18:26:41.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.68 --image-format 2 -s 1 2023-09-29T18:26:41.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.804 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.69 --image-format 2 -s 1 2023-09-29T18:26:41.847 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.847 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.70 --image-format 2 -s 1 2023-09-29T18:26:41.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.71 --image-format 2 -s 1 2023-09-29T18:26:41.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.72 --image-format 2 -s 1 2023-09-29T18:26:41.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:41.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.73 --image-format 2 -s 1 2023-09-29T18:26:42.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.024 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.74 --image-format 2 -s 1 2023-09-29T18:26:42.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.75 --image-format 2 -s 1 2023-09-29T18:26:42.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.76 --image-format 2 -s 1 2023-09-29T18:26:42.164 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.164 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.77 --image-format 2 -s 1 2023-09-29T18:26:42.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.78 --image-format 2 -s 1 2023-09-29T18:26:42.273 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.273 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.79 --image-format 2 -s 1 2023-09-29T18:26:42.314 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.314 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.80 --image-format 2 -s 1 2023-09-29T18:26:42.354 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.355 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.81 --image-format 2 -s 1 2023-09-29T18:26:42.397 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.397 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.82 --image-format 2 -s 1 2023-09-29T18:26:42.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.440 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.83 --image-format 2 -s 1 2023-09-29T18:26:42.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.84 --image-format 2 -s 1 2023-09-29T18:26:42.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.533 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.85 --image-format 2 -s 1 2023-09-29T18:26:42.588 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.589 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.86 --image-format 2 -s 1 2023-09-29T18:26:42.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.87 --image-format 2 -s 1 2023-09-29T18:26:42.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.88 --image-format 2 -s 1 2023-09-29T18:26:42.721 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.721 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.89 --image-format 2 -s 1 2023-09-29T18:26:42.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.90 --image-format 2 -s 1 2023-09-29T18:26:42.808 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.808 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.91 --image-format 2 -s 1 2023-09-29T18:26:42.856 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.856 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.92 --image-format 2 -s 1 2023-09-29T18:26:42.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.93 --image-format 2 -s 1 2023-09-29T18:26:42.940 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.940 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.94 --image-format 2 -s 1 2023-09-29T18:26:42.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:42.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.95 --image-format 2 -s 1 2023-09-29T18:26:43.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.96 --image-format 2 -s 1 2023-09-29T18:26:43.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.97 --image-format 2 -s 1 2023-09-29T18:26:43.107 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.107 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.98 --image-format 2 -s 1 2023-09-29T18:26:43.155 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.155 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create image.99 --image-format 2 -s 1 2023-09-29T18:26:43.202 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:43.202 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:43.203 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 100 2023-09-29T18:26:43.230 INFO:tasks.workunit.client.0.smithi143.stdout:100 2023-09-29T18:26:43.230 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:26:43.231 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep image 2023-09-29T18:26:43.231 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:43.231 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 100 2023-09-29T18:26:43.284 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.283+0000 7fb64851f700 0 -- 172.21.15.143:0/386370285 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5640269ed760 msgr2=0x5640269e8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.287 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.287+0000 7fb647d1e700 0 -- 172.21.15.143:0/386370285 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x564026a95c90 msgr2=0x564026a9e810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.552 INFO:tasks.workunit.client.0.smithi143.stdout:100 2023-09-29T18:26:43.553 INFO:tasks.workunit.client.0.smithi143.stderr:++ seq -w 00 99 2023-09-29T18:26:43.553 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.553 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.00 2023-09-29T18:26:43.642 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.641+0000 7f44f8f78700 0 -- 172.21.15.143:0/2620274522 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f44d8005ff0 msgr2=0x7f44d8008470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.642 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:43.644 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.644 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.01 2023-09-29T18:26:43.774 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.774+0000 7f598928d700 0 -- 172.21.15.143:0/2632521044 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5964008740 msgr2=0x7f596400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.775 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:43.778 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.778 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.02 2023-09-29T18:26:43.866 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.865+0000 7fd85842c700 0 -- 172.21.15.143:0/1032053827 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fd83c008740 msgr2=0x7fd83c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.866 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:43.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.03 2023-09-29T18:26:43.959 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:43.959+0000 7f721a322700 0 -- 172.21.15.143:0/961037099 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557df4f8db90 msgr2=0x557df4f424c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:43.960 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:43.962 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:43.963 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.04 2023-09-29T18:26:44.079 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.079+0000 7f5e480e1700 0 -- 172.21.15.143:0/1068533908 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557b99462cc0 msgr2=0x557b99465e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:44.081 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.084 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.084 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.05 2023-09-29T18:26:44.177 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.177+0000 7fdc16844700 0 -- 172.21.15.143:0/759553666 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fdbf4008740 msgr2=0x7fdbf400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:44.178 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.06 2023-09-29T18:26:44.299 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.298+0000 7f6b748ef700 0 -- 172.21.15.143:0/512475932 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5643cf04b010 msgr2=0x5643cf06c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:44.299 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.07 2023-09-29T18:26:44.604 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.603+0000 7fd7147f3700 0 -- 172.21.15.143:0/1535695432 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x56386a2832f0 msgr2=0x56386a294cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:44.611 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.08 2023-09-29T18:26:44.719 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.719+0000 7fae2acfb700 0 -- 172.21.15.143:0/2921794102 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fae08008740 msgr2=0x7fae0800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:44.720 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.723 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.723 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.09 2023-09-29T18:26:44.825 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.824+0000 7fa816688700 0 -- 172.21.15.143:0/3569740749 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fa7fc008740 msgr2=0x7fa7fc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:44.825 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.827 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.827 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.10 2023-09-29T18:26:44.935 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:44.934+0000 7f777a37c700 0 -- 172.21.15.143:0/1982557472 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7760006500 msgr2=0x7f7760011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:44.935 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:44.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:44.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.11 2023-09-29T18:26:45.028 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.028+0000 7f954180b700 0 -- 172.21.15.143:0/855378116 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5600a7a1e560 msgr2=0x5600a7a449f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.029 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.033 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.033 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.12 2023-09-29T18:26:45.121 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.120+0000 7f9e7a38c700 0 -- 172.21.15.143:0/2688316064 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f9e5c03a3c0 msgr2=0x7f9e5c03c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.131 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.13 2023-09-29T18:26:45.226 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.225+0000 7fd5f838b700 0 -- 172.21.15.143:0/3483127069 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fd5dc03a3c0 msgr2=0x7fd5dc03c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:45.232 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.235 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.236 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.14 2023-09-29T18:26:45.325 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.324+0000 7fbfcdc85700 0 -- 172.21.15.143:0/532808823 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557eecdd77d0 msgr2=0x557eecdd8730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.325 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.327 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.327 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.15 2023-09-29T18:26:45.433 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.433+0000 7f624df72700 0 -- 172.21.15.143:0/1483714170 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f623003a3c0 msgr2=0x7f623003c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.439 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.442 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.442 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.16 2023-09-29T18:26:45.537 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.537+0000 7f4bbd57e700 0 -- 172.21.15.143:0/3849946022 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557398445c90 msgr2=0x55739841e330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.538 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.17 2023-09-29T18:26:45.713 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.713+0000 7f71d87e4700 0 -- 172.21.15.143:0/1531429449 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f71b4008740 msgr2=0x7f71b400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.714 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.18 2023-09-29T18:26:45.814 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.813+0000 7f5321775700 0 -- 172.21.15.143:0/685573685 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f52fc006500 msgr2=0x7f52fc011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.814 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.817 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.817 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.19 2023-09-29T18:26:45.930 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:45.929+0000 7f7546249700 0 -- 172.21.15.143:0/1516363357 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55a748812370 msgr2=0x55a748833f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:45.930 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:45.932 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:45.933 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.20 2023-09-29T18:26:46.033 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.033+0000 7f3bfbcd0700 0 -- 172.21.15.143:0/3342198710 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f3be0008740 msgr2=0x7f3be000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.034 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.037 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.037 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.21 2023-09-29T18:26:46.124 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.124+0000 7fdbea551700 0 -- 172.21.15.143:0/2195066803 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fdbcc03a370 msgr2=0x7fdbcc03c770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.129 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.132 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.132 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.22 2023-09-29T18:26:46.237 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.240 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.23 2023-09-29T18:26:46.353 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.353+0000 7fae90b0d700 0 -- 172.21.15.143:0/1810743242 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fae74008740 msgr2=0x7fae7400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.354 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.356 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.357 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.24 2023-09-29T18:26:46.456 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.455+0000 7fcfb0ee6700 0 -- 172.21.15.143:0/508073093 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x564810851d40 msgr2=0x564810807450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.456 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.459 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.459 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.25 2023-09-29T18:26:46.560 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.560+0000 7fcfb65b4700 0 -- 172.21.15.143:0/413548346 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fcf9c008740 msgr2=0x7fcf9c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:46.563 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.566 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.566 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.26 2023-09-29T18:26:46.687 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.686+0000 7f6ce28f6700 0 -- 172.21.15.143:0/2380061552 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f6cc4005fb0 msgr2=0x7f6cc4008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.689 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.690 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.690 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.27 2023-09-29T18:26:46.782 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.782+0000 7f841c4a9700 0 -- 172.21.15.143:0/141700863 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5629894e7d10 msgr2=0x562989509ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:46.783 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.786 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.786 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.28 2023-09-29T18:26:46.897 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:46.897+0000 7f6152085700 0 -- 172.21.15.143:0/2322349310 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x559abf80f3d0 msgr2=0x559abf7e71c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:46.898 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:46.900 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:46.900 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.29 2023-09-29T18:26:47.015 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.015+0000 7f2ff151a700 0 -- 172.21.15.143:0/2438720640 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f2fcc008740 msgr2=0x7f2fcc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.016 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.019 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.019 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.30 2023-09-29T18:26:47.109 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.109+0000 7f5012ae2700 0 -- 172.21.15.143:0/918269174 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f4ff8008740 msgr2=0x7f4ff800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.109 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.112 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.112 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.31 2023-09-29T18:26:47.222 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.221+0000 7ffbe36c2700 0 -- 172.21.15.143:0/1024306953 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffbc00065b0 msgr2=0x7ffbc0011ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:47.222 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.225 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.225 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.32 2023-09-29T18:26:47.331 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.330+0000 7fe2d7ab4700 0 -- 172.21.15.143:0/9793849 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5623c5659b50 msgr2=0x5623c5630590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.331 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.334 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.335 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.33 2023-09-29T18:26:47.452 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.452+0000 7ff049db9700 0 -- 172.21.15.143:0/2939916021 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ff030008740 msgr2=0x7ff03000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.453 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.456 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.456 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.34 2023-09-29T18:26:47.564 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.564+0000 7fe4e218f700 0 -- 172.21.15.143:0/759654133 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fe4c8008740 msgr2=0x7fe4c800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:47.565 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.35 2023-09-29T18:26:47.872 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.872+0000 7fd611280700 0 -- 172.21.15.143:0/1317203470 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5559d240c7b0 msgr2=0x5559d24b25f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.882 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.885 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.36 2023-09-29T18:26:47.974 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:47.974+0000 7fe433ace700 0 -- 172.21.15.143:0/1505179575 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fe410008740 msgr2=0x7fe41000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:47.975 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:47.978 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:47.978 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.37 2023-09-29T18:26:48.130 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.129+0000 7f5c2aa99700 0 -- 172.21.15.143:0/1523074065 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5c10008740 msgr2=0x7f5c1000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.130 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.38 2023-09-29T18:26:48.231 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.230+0000 7fb3b9978700 0 -- 172.21.15.143:0/4073549864 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fb398008740 msgr2=0x7fb39800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.231 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.234 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.234 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.39 2023-09-29T18:26:48.406 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.405+0000 7f411fa9f700 0 -- 172.21.15.143:0/2508759462 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f40fc008740 msgr2=0x7f40fc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.406 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.408 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.409 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.40 2023-09-29T18:26:48.512 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.511+0000 7f6732372700 0 -- 172.21.15.143:0/2839760699 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55c75bf29210 msgr2=0x55c75bf3a6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.513 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.516 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.516 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.41 2023-09-29T18:26:48.709 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.708+0000 7f75a8b51700 0 -- 172.21.15.143:0/352676694 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7584008740 msgr2=0x7f758400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.709 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.42 2023-09-29T18:26:48.822 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.821+0000 7f8085a9d700 0 -- 172.21.15.143:0/4134334732 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557ef36fdba0 msgr2=0x557ef3700170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.822 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.43 2023-09-29T18:26:48.981 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:48.981+0000 7f47de16a700 0 -- 172.21.15.143:0/3121186035 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5561793a3a10 msgr2=0x5561793c5e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:48.982 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:48.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:48.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.44 2023-09-29T18:26:49.089 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.088+0000 7f6771850700 0 -- 172.21.15.143:0/1890787509 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55ab1302aea0 msgr2=0x55ab1302b270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.089 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.45 2023-09-29T18:26:49.190 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.189+0000 7fe9df57d700 0 -- 172.21.15.143:0/2829983224 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fe9bc008740 msgr2=0x7fe9bc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.190 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.192 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.193 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.46 2023-09-29T18:26:49.285 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.284+0000 7f6f44958700 0 -- 172.21.15.143:0/1382117422 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f6f20008740 msgr2=0x7f6f2000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.285 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.288 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.289 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.47 2023-09-29T18:26:49.382 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.381+0000 7f5b4d33c700 0 -- 172.21.15.143:0/2724394693 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5b2c005fb0 msgr2=0x7f5b2c008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.382 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.385 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.385 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.48 2023-09-29T18:26:49.514 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.513+0000 7f486a757700 0 -- 172.21.15.143:0/1912033165 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f4850008740 msgr2=0x7f485000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.514 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.517 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.517 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.49 2023-09-29T18:26:49.620 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.619+0000 7fe5f6cdd700 0 -- 172.21.15.143:0/434961005 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fe5d4008740 msgr2=0x7fe5d400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.620 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.623 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.623 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.50 2023-09-29T18:26:49.741 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.740+0000 7fb7264c1700 0 -- 172.21.15.143:0/1784049433 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fb70c008740 msgr2=0x7fb70c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.741 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.51 2023-09-29T18:26:49.847 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.846+0000 7f5c8fdd1700 0 -- 172.21.15.143:0/707422981 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5c6c008740 msgr2=0x7f5c6c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.847 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.52 2023-09-29T18:26:49.937 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:49.937+0000 7f76a2cb6700 0 -- 172.21.15.143:0/2609961168 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x56186132fbd0 msgr2=0x5618613d0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:49.943 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:49.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:49.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.53 2023-09-29T18:26:50.076 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.075+0000 7f14e9019700 0 -- 172.21.15.143:0/974072293 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f14c4008740 msgr2=0x7f14c400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.076 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.54 2023-09-29T18:26:50.172 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.171+0000 7f1ef6918700 0 -- 172.21.15.143:0/461974709 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55c1f16957b0 msgr2=0x55c1f173b5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.173 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.177 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.177 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.55 2023-09-29T18:26:50.278 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.278+0000 7fa373926700 0 -- 172.21.15.143:0/2978662333 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55578a60efd0 msgr2=0x55578a630ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.278 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.281 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.281 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.56 2023-09-29T18:26:50.387 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.386+0000 7fbd460eb700 0 -- 172.21.15.143:0/331143130 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fbd2803a440 msgr2=0x7fbd2803c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.395 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.398 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.398 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.57 2023-09-29T18:26:50.493 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.493+0000 7f6a575e7700 0 -- 172.21.15.143:0/2979830302 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f6a34006500 msgr2=0x7f6a34011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.494 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.496 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.497 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.58 2023-09-29T18:26:50.605 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.605+0000 7f77df842700 0 -- 172.21.15.143:0/2124004489 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f77c0005fd0 msgr2=0x7f77c0008450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:50.807 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.810 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.810 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.59 2023-09-29T18:26:50.903 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:50.903+0000 7f0798a7f700 0 -- 172.21.15.143:0/1990501266 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55794f745a80 msgr2=0x55794f78de40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:50.903 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:50.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:50.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.60 2023-09-29T18:26:51.001 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.001+0000 7f28111d8700 0 -- 172.21.15.143:0/548727720 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557b0eb96b10 msgr2=0x557b0ebb8ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:51.002 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.004 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.005 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.61 2023-09-29T18:26:51.120 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.120+0000 7f93f547d700 0 -- 172.21.15.143:0/240927444 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f93d0008740 msgr2=0x7f93d000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.121 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.123 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.124 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.62 2023-09-29T18:26:51.236 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.236+0000 7f0c7bac4700 0 -- 172.21.15.143:0/131328549 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x563592fc9b10 msgr2=0x563592fa14c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.237 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.63 2023-09-29T18:26:51.332 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.332+0000 7efe674fb700 0 -- 172.21.15.143:0/3576170231 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7efe44008740 msgr2=0x7efe4400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.332 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.335 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.335 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.64 2023-09-29T18:26:51.434 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.433+0000 7fd8ac531700 0 -- 172.21.15.143:0/2199370424 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x556d40955a60 msgr2=0x556d40958500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.434 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.437 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.437 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.65 2023-09-29T18:26:51.542 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.541+0000 7ff425537700 0 -- 172.21.15.143:0/1023926190 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ff400008740 msgr2=0x7ff40000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.542 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.545 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.545 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.66 2023-09-29T18:26:51.647 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.647+0000 7f783ed1c700 0 -- 172.21.15.143:0/679563150 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f781c008740 msgr2=0x7f781c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.647 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.650 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.650 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.67 2023-09-29T18:26:51.744 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.744+0000 7f6a10480700 0 -- 172.21.15.143:0/2124695521 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55f7f86fea00 msgr2=0x55f7f86b4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.745 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.748 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.748 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.68 2023-09-29T18:26:51.861 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.860+0000 7fa5f2e4d700 0 -- 172.21.15.143:0/279767961 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fa5cc005fd0 msgr2=0x7fa5cc008450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.861 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.69 2023-09-29T18:26:51.965 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:51.965+0000 7efe86efd700 0 -- 172.21.15.143:0/3103938950 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7efe64006510 msgr2=0x7efe64011ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:51.966 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:51.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:51.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.70 2023-09-29T18:26:52.054 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.053+0000 7f4d75d97700 0 -- 172.21.15.143:0/3976225383 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f4d5803a370 msgr2=0x7f4d5803c770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.061 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.71 2023-09-29T18:26:52.175 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.174+0000 7f7cebb49700 0 -- 172.21.15.143:0/2363483287 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7ccc03a3c0 msgr2=0x7f7ccc03c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.185 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.187 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.187 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.72 2023-09-29T18:26:52.281 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.281+0000 7f58df57d700 0 -- 172.21.15.143:0/3262931999 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f58c0005fb0 msgr2=0x7f58c0008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.282 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.284 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.284 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.73 2023-09-29T18:26:52.378 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.378+0000 7f41eda66700 0 -- 172.21.15.143:0/2512691879 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x555e20d432b0 msgr2=0x555e20d40400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.379 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.382 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.382 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.74 2023-09-29T18:26:52.491 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.490+0000 7f10e2cbd700 0 -- 172.21.15.143:0/2096014359 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f10c8008740 msgr2=0x7f10c800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.491 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.494 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.494 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.75 2023-09-29T18:26:52.589 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.589+0000 7f7b26be2700 0 -- 172.21.15.143:0/1885112104 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x559a9c101a30 msgr2=0x559a9c122c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.590 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.592 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.76 2023-09-29T18:26:52.693 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.693+0000 7f73397c8700 0 -- 172.21.15.143:0/3710631195 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55a46f7af7b0 msgr2=0x55a46f7d1980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.694 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.697 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.697 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.77 2023-09-29T18:26:52.815 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.815+0000 7f7197955700 0 -- 172.21.15.143:0/2016853129 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7174008740 msgr2=0x7f717400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.816 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.818 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.818 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.78 2023-09-29T18:26:52.910 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:52.909+0000 7f40c8a5c700 0 -- 172.21.15.143:0/3947881803 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f40a4008740 msgr2=0x7f40a400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:52.910 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:52.913 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:52.913 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.79 2023-09-29T18:26:53.015 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.015+0000 7f449fc0d700 0 -- 172.21.15.143:0/1832288101 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x561d99558c40 msgr2=0x561d9950e340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.016 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.019 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.019 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.80 2023-09-29T18:26:53.154 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.154+0000 7f7605c34700 0 -- 172.21.15.143:0/2315117028 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f75e803a3c0 msgr2=0x7f75e803c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.164 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.167 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.167 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.81 2023-09-29T18:26:53.267 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.266+0000 7ff742ea3700 0 -- 172.21.15.143:0/2494118689 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ff720008740 msgr2=0x7ff72000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.267 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.82 2023-09-29T18:26:53.398 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.398+0000 7f46fdeae700 0 -- 172.21.15.143:0/812236259 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f46e003a3c0 msgr2=0x7f46e003c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.426 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.429 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.429 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.83 2023-09-29T18:26:53.546 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.546+0000 7f8fdacdd700 0 -- 172.21.15.143:0/2643789688 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f8fb8008740 msgr2=0x7f8fb800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.547 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.550 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.550 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.84 2023-09-29T18:26:53.661 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.659+0000 7f24908c9700 0 -- 172.21.15.143:0/2428496779 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f246c006500 msgr2=0x7f246c011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.661 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.662 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.663 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.85 2023-09-29T18:26:53.775 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.775+0000 7f94bfdd3700 0 -- 172.21.15.143:0/2671000904 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5605f6e72d40 msgr2=0x5605f6e75e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.776 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.779 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.779 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.86 2023-09-29T18:26:53.872 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.871+0000 7f8fcdc85700 0 -- 172.21.15.143:0/3773881772 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f8fb4008740 msgr2=0x7f8fb400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:53.872 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.87 2023-09-29T18:26:53.994 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:53.993+0000 7f2b4f45b700 0 -- 172.21.15.143:0/1748386284 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x562656686a40 msgr2=0x562656664050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:53.994 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:53.996 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:53.996 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.88 2023-09-29T18:26:54.097 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.097+0000 7f1044911700 0 -- 172.21.15.143:0/3924402173 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f1020008740 msgr2=0x7f102000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.097 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.89 2023-09-29T18:26:54.215 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.215+0000 7feb466a9700 0 -- 172.21.15.143:0/3418695702 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7feb2c008740 msgr2=0x7feb2c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:54.216 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.219 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.219 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.90 2023-09-29T18:26:54.346 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.346+0000 7f96c8735700 0 -- 172.21.15.143:0/199888942 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55cd26d88e50 msgr2=0x55cd26daac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.347 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.349 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.350 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.91 2023-09-29T18:26:54.450 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.450+0000 7f508dc13700 0 -- 172.21.15.143:0/4234625812 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5074008740 msgr2=0x7f507400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.451 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.454 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.454 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.92 2023-09-29T18:26:54.563 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.563+0000 7f17ae1b8700 0 -- 172.21.15.143:0/400353171 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f17940065b0 msgr2=0x7f1794011ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.564 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.567 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.567 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.93 2023-09-29T18:26:54.677 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.676+0000 7f466dbf0700 0 -- 172.21.15.143:0/2429974556 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x562e3c14f1d0 msgr2=0x562e3c1757f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.677 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.680 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.680 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.94 2023-09-29T18:26:54.789 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.789+0000 7f151efb4700 0 -- 172.21.15.143:0/3488552859 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f1504008740 msgr2=0x7f150400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:54.790 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.95 2023-09-29T18:26:54.889 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.889+0000 7f72b636c700 0 -- 172.21.15.143:0/3604741024 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557b29ed55d0 msgr2=0x557b29ed59a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:54.889 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:54.892 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:54.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.96 2023-09-29T18:26:54.992 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:54.991+0000 7fa24a4ae700 0 -- 172.21.15.143:0/827692527 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fa22c03a3c0 msgr2=0x7fa22c03c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:26:55.006 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:55.008 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:55.009 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.97 2023-09-29T18:26:55.107 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:55.106+0000 7efc0da4d700 0 -- 172.21.15.143:0/3034717439 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7efbf4006500 msgr2=0x7efbf4011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:55.107 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:55.110 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:55.110 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.98 2023-09-29T18:26:55.214 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:55.213+0000 7f2cdc915700 0 -- 172.21.15.143:0/1504654655 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55c6a1a94c50 msgr2=0x55c6a1ab6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:55.214 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:55.217 INFO:tasks.workunit.client.0.smithi143.stderr:+ for i in $(seq -w 00 99) 2023-09-29T18:26:55.217 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm image.99 2023-09-29T18:26:55.326 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:55.326+0000 7f4d2e0bc700 0 -- 172.21.15.143:0/2297987065 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55adc25b2bd0 msgr2=0x55adc2653920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:55.328 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:55.331 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_remove 2023-09-29T18:26:55.331 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing remove...' 2023-09-29T18:26:55.332 INFO:tasks.workunit.client.0.smithi143.stdout:testing remove... 2023-09-29T18:26:55.332 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:55.332 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.505 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.585 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.662 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.736 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.816 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:55.975 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.051 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.126 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.202 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.357 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.436 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.518 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.594 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.670 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:56.749 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd remove NOT_EXIST 2023-09-29T18:26:56.793 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 0% complete...failed. 2023-09-29T18:26:56.793 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: delete error: (2) No such file or directory 2023-09-29T18:26:56.796 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:26:56.796 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 test1 2023-09-29T18:26:56.819 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:56.825 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:56.824+0000 7fcdeaeec500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:56.832 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:26:56.873 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:56.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:56.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:56.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:26:56.902 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:56.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test2 2023-09-29T18:26:56.947 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:26:57.046 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:57.046+0000 7f7f44663700 0 -- 172.21.15.143:0/3095935581 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7f20008740 msgr2=0x7f7f2000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:57.047 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:57.049 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:57.049 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:57.050 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:26:57.076 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:57.077 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 1 test1 2023-09-29T18:26:57.099 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:26:57.105 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:57.105+0000 7f0f5daba500 -1 librbd: Forced V1 image creation. 2023-09-29T18:26:57.113 INFO:tasks.workunit.client.0.smithi143.stderr:+ rados rm -p rbd test1.rbd 2023-09-29T18:26:57.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:26:57.172 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:57.174 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:57.174 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:57.174 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:26:57.200 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:57.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 1 -eq 0 ']' 2023-09-29T18:26:57.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test2 2023-09-29T18:26:57.245 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test2@snap 2023-09-29T18:26:57.650 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:26:57.657 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect test2@snap 2023-09-29T18:26:57.698 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test2@snap clone --rbd-default-clone-format 1 2023-09-29T18:26:57.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ rados -p rbd rm rbd_children 2023-09-29T18:26:57.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm clone 2023-09-29T18:26:57.902 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:57.902+0000 7fc4fd98f700 0 -- 172.21.15.143:0/3657444370 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557f37022c50 msgr2=0x557f37021430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:57.902 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:57.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:26:57.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep clone 2023-09-29T18:26:57.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:26:57.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:26:57.933 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:26:57.933 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect test2@snap 2023-09-29T18:26:57.974 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm test2@snap 2023-09-29T18:26:58.655 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:26:58.666 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:26:58.778 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:26:58.776+0000 7f4a9b2e3700 0 -- 172.21.15.143:0/847310496 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f4a80008740 msgr2=0x7f4a8000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:26:58.778 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:26:58.779 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_migration 2023-09-29T18:26:58.779 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing migration...' 2023-09-29T18:26:58.780 INFO:tasks.workunit.client.0.smithi143.stdout:testing migration... 2023-09-29T18:26:58.780 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:26:58.781 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:58.854 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:58.929 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.002 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.078 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.156 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.235 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.313 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.591 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.677 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.752 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.911 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:26:59.987 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:00.062 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:00.135 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:00.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:00.289 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:00.368 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool create rbd2 8 2023-09-29T18:27:01.776 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' already exists 2023-09-29T18:27:01.785 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd pool init rbd2 2023-09-29T18:27:04.668 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 1 -s 128M test1 2023-09-29T18:27:04.693 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:04.700 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:04.700+0000 7f8f23408500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:04.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:04.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'format: 1' 2023-09-29T18:27:04.747 INFO:tasks.workunit.client.0.smithi143.stdout: format: 1 2023-09-29T18:27:04.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 --image-format 2 2023-09-29T18:27:04.822 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state test1 2023-09-29T18:27:04.822 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=test1 2023-09-29T18:27:04.822 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status test1 2023-09-29T18:27:04.823 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:04.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ test prepared = prepared 2023-09-29T18:27:04.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:04.904 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'format: 2' 2023-09-29T18:27:04.940 INFO:tasks.workunit.client.0.smithi143.stdout: format: 2 2023-09-29T18:27:04.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:04.975 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:04.974+0000 7f5658403500 -1 librbd::image::PreRemoveRequest: 0x559cbe23c8f0 validate_image_removal: image in migration state - not removing 2023-09-29T18:27:04.977 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 0% complete...failed. 2023-09-29T18:27:04.977 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: error: image still has watchers 2023-09-29T18:27:04.977 INFO:tasks.workunit.client.0.smithi143.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-09-29T18:27:04.980 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:04.980 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:05.090 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:05.090+0000 7f5e6cd26700 0 -- 172.21.15.143:0/139242323 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f5e4c004b70 msgr2=0x7f5e4c006f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.112 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:05.112+0000 7f5e6d527700 0 -- 172.21.15.143:0/139242323 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x559fa3822b50 msgr2=0x559fa3914c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.136 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:05.141 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state test1 2023-09-29T18:27:05.141 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=test1 2023-09-29T18:27:05.141 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status test1 2023-09-29T18:27:05.141 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:05.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ test executed = executed 2023-09-29T18:27:05.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:05.289 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete...2023-09-29T18:27:05.289+0000 7feef26e5700 0 -- 172.21.15.143:0/2119336670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55c604431670 msgr2=0x55c60448fd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.302 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-09-29T18:27:05.301+0000 7feef1ee4700 0 -- 172.21.15.143:0/2119336670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7feecc004bb0 msgr2=0x7feecc006f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.312 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:05.315 INFO:tasks.workunit.client.0.smithi143.stderr:+ get_migration_state test1 2023-09-29T18:27:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ local image=test1 2023-09-29T18:27:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --format xml status test1 2023-09-29T18:27:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:05.351 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:05.353 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:05.353 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'features: .*layering' 2023-09-29T18:27:05.384 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:05.385 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 --image-feature layering,exclusive-lock,object-map,fast-diff,deep-flatten 2023-09-29T18:27:05.497 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:05.498 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'features: .*layering' 2023-09-29T18:27:05.537 INFO:tasks.workunit.client.0.smithi143.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, migrating 2023-09-29T18:27:05.537 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:05.614 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:05.614+0000 7ff042bb7700 0 -- 172.21.15.143:0/1443520614 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7ff01c004be0 msgr2=0x7ff01c006fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.635 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:05.635+0000 7ff0433b8700 0 -- 172.21.15.143:0/1443520614 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5639146fd930 msgr2=0x5639147ee8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.639 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:05.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:05.726 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:05.726+0000 7f99016a9700 0 -- 172.21.15.143:0/339471793 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f98e0004bb0 msgr2=0x7f98e0006f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.737 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete...2023-09-29T18:27:05.737+0000 7f9901eaa700 0 -- 172.21.15.143:0/339471793 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x558a8e4a9680 msgr2=0x558a8e59fb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.777 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete... Commit image migration: 100% complete...done. 2023-09-29T18:27:05.780 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 rbd2/test1 2023-09-29T18:27:05.882 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:05.881+0000 7fc62f90f700 0 -- 172.21.15.143:0/846203199 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fc610004be0 msgr2=0x7fc610006fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:05.892 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state rbd2/test1 2023-09-29T18:27:05.892 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=rbd2/test1 2023-09-29T18:27:05.892 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status rbd2/test1 2023-09-29T18:27:05.892 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:05.964 INFO:tasks.workunit.client.0.smithi143.stderr:+ test prepared = prepared 2023-09-29T18:27:05.964 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:27:05.964 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:05.964 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:27:05.990 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:27:05.991 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls 2023-09-29T18:27:05.991 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:27:06.019 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:27:06.019 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:06.116 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:06.116+0000 7f58064c0700 0 -- 172.21.15.143:0/4010080555 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x558b3e5c77b0 msgr2=0x558b3e6bc870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.127 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:06.130 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state rbd2/test1 2023-09-29T18:27:06.130 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=rbd2/test1 2023-09-29T18:27:06.131 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status rbd2/test1 2023-09-29T18:27:06.131 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:06.192 INFO:tasks.workunit.client.0.smithi143.stderr:+ test executed = executed 2023-09-29T18:27:06.192 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm rbd2/test1 2023-09-29T18:27:06.228 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:06.227+0000 7f8616ffd700 -1 librbd::image::PreRemoveRequest: 0x563dbb89a890 validate_image_removal: image in migration state - not removing 2023-09-29T18:27:06.233 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 0% complete...failed. 2023-09-29T18:27:06.233 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: error: image still has watchers 2023-09-29T18:27:06.233 INFO:tasks.workunit.client.0.smithi143.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-09-29T18:27:06.235 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:06.236 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:06.318 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-09-29T18:27:06.318+0000 7f9cdf640700 0 -- 172.21.15.143:0/2335987847 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x559628706680 msgr2=0x5596287fcb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.331 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:06.334 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create rbd2/ns1 2023-09-29T18:27:06.365 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create rbd2/ns2 2023-09-29T18:27:06.395 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare rbd2/test1 rbd2/ns1/test1 2023-09-29T18:27:06.481 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:06.481+0000 7fc4c5a69700 0 -- 172.21.15.143:0/3659610623 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x564a24e20fa0 msgr2=0x564a24ec28c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.488 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state rbd2/ns1/test1 2023-09-29T18:27:06.488 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=rbd2/ns1/test1 2023-09-29T18:27:06.488 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-09-29T18:27:06.488 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:06.560 INFO:tasks.workunit.client.0.smithi143.stderr:+ test prepared = prepared 2023-09-29T18:27:06.560 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute rbd2/test1 2023-09-29T18:27:06.637 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:06.637+0000 7f89cb180700 0 -- 172.21.15.143:0/834672230 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55c3229da610 msgr2=0x55c322a38c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.641 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:06.644 INFO:tasks.workunit.client.0.smithi143.stderr:++ get_migration_state rbd2/ns1/test1 2023-09-29T18:27:06.644 INFO:tasks.workunit.client.0.smithi143.stderr:++ local image=rbd2/ns1/test1 2023-09-29T18:27:06.644 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-09-29T18:27:06.644 INFO:tasks.workunit.client.0.smithi143.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-09-29T18:27:06.703 INFO:tasks.workunit.client.0.smithi143.stderr:+ test executed = executed 2023-09-29T18:27:06.703 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit rbd2/test1 2023-09-29T18:27:06.780 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-09-29T18:27:06.780+0000 7f4fc4cbb700 0 -- 172.21.15.143:0/3180488279 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x56119187c0e0 msgr2=0x56119187e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.794 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:06.797 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare rbd2/ns1/test1 rbd2/ns2/test1 2023-09-29T18:27:06.894 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:06.894+0000 7f463e0a0700 0 -- 172.21.15.143:0/2457796855 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x561509640fa0 msgr2=0x5615096e28c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute rbd2/ns2/test1 2023-09-29T18:27:06.968 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:06.967+0000 7f76eb3ed700 0 -- 172.21.15.143:0/139112810 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5578aa0cd220 msgr2=0x5578aa0cf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.982 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:06.981+0000 7f76eb3ed700 0 -- 172.21.15.143:0/139112810 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f76d003ac10 msgr2=0x7f76d003d030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:06.984 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:06.987 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit rbd2/ns2/test1 2023-09-29T18:27:07.064 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.063+0000 7fb697cad700 0 -- 172.21.15.143:0/370840843 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55924f6ed5f0 msgr2=0x55924f6ef9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.073 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.072+0000 7fb6984ae700 0 -- 172.21.15.143:0/370840843 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55924f5f5680 msgr2=0x55924f69feb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.089 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete... Commit image migration: 100% complete...done. 2023-09-29T18:27:07.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M test1 2023-09-29T18:27:07.119 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.118+0000 7fbc464e3500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:07.126 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 --data-pool rbd2 2023-09-29T18:27:07.203 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:07.203 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'data_pool: rbd2' 2023-09-29T18:27:07.241 INFO:tasks.workunit.client.0.smithi143.stdout: data_pool: rbd2 2023-09-29T18:27:07.241 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:07.322 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:07.321+0000 7f5e363ac700 0 -- 172.21.15.143:0/2093184833 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5586dfb99610 msgr2=0x5586dfbf7730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.338 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.337+0000 7f5e363ac700 0 -- 172.21.15.143:0/2093184833 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5e1c008740 msgr2=0x7f5e1803d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.354 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:07.357 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:07.443 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete...2023-09-29T18:27:07.443+0000 7f5e1dcea700 0 -- 172.21.15.143:0/3641164827 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5650c2b9e680 msgr2=0x5650c2c94b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:07.453 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-09-29T18:27:07.453+0000 7f5e1dcea700 0 -- 172.21.15.143:0/3641164827 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5e0003ab40 msgr2=0x7f5e0003cf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.466 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:07.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 2023-09-29T18:27:07.559 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd trash mv test1 2023-09-29T18:27:07.559 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv test1 2023-09-29T18:27:07.598 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.597+0000 7f4ac069d500 -1 librbd::api::Trash: move: cannot move migrating image to trash 2023-09-29T18:27:07.604 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: deferred delete error: (16) Device or resource busy 2023-09-29T18:27:07.606 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:07.607 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls -a 2023-09-29T18:27:07.607 INFO:tasks.workunit.client.0.smithi143.stderr:++ cut -d ' ' -f 1 2023-09-29T18:27:07.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=19f49dd23be6 2023-09-29T18:27:07.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd trash rm 19f49dd23be6 2023-09-29T18:27:07.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm 19f49dd23be6 2023-09-29T18:27:07.668 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.667+0000 7f248a80d700 -1 librbd::image::RefreshRequest: image being migrated 2023-09-29T18:27:07.668 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.667+0000 7f248a80d700 -1 librbd::image::OpenRequest: failed to refresh image: (30) Read-only file system 2023-09-29T18:27:07.668 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.667+0000 7f248a80d700 -1 librbd::ImageState: 0x7f246c04d9c0 failed to open image: (30) Read-only file system 2023-09-29T18:27:07.668 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.667+0000 7f24797fa700 -1 librbd::image::RemoveRequest: 0x7f246c000b80 handle_open_image: error opening image: (30) Read-only file system 2023-09-29T18:27:07.669 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: remove error: (30) Read-only file system 2023-09-29T18:27:07.670 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 0% complete...failed. 2023-09-29T18:27:07.672 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:07.672 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd trash restore 19f49dd23be6 2023-09-29T18:27:07.672 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash restore 19f49dd23be6 2023-09-29T18:27:07.698 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: restore error: 2023-09-29T18:27:07.697+0000 7f186aadd500 -1 librbd::api::Trash: restore: Current trash source 'migration' does not match expected: user,mirroring,unknown (4) 2023-09-29T18:27:07.698 INFO:tasks.workunit.client.0.smithi143.stderr:(22) Invalid argument 2023-09-29T18:27:07.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:07.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test1 2023-09-29T18:27:07.799 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:07.797+0000 7fbf2259f700 0 -- 172.21.15.143:0/1356407659 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55f527ee7610 msgr2=0x55f527f45710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.818 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:07.818+0000 7fbf21d9e700 0 -- 172.21.15.143:0/1356407659 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fbefc004be0 msgr2=0x7fbefc006fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:07.824 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:07.827 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd remove test1 2023-09-29T18:27:07.919 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:07.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/dev/urandom bs=1M count=1 2023-09-29T18:27:07.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd --image-format 2 import - test1 2023-09-29T18:27:07.972 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records in 2023-09-29T18:27:07.972 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records out 2023-09-29T18:27:07.973 INFO:tasks.workunit.client.0.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0498747 s, 21.0 MB/s 2023-09-29T18:27:08.007 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 100% complete...done. 2023-09-29T18:27:08.010 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export test1 - 2023-09-29T18:27:08.010 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:08.205 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:08.207 INFO:tasks.workunit.client.0.smithi143.stderr:+ md5sum='44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:08.207 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@snap1 2023-09-29T18:27:08.675 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:27:08.685 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect test1@snap1 2023-09-29T18:27:08.725 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@snap2 2023-09-29T18:27:09.679 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:27:09.691 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test1@snap1 clone_v1 --rbd_default_clone_format=1 2023-09-29T18:27:09.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test1@snap2 clone_v2 --rbd_default_clone_format=2 2023-09-29T18:27:09.844 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v1 2023-09-29T18:27:09.845 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-09-29T18:27:09.885 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd/test1@snap1 2023-09-29T18:27:09.886 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:09.886 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-09-29T18:27:09.927 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd/test1@snap2 2023-09-29T18:27:09.927 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:09.927 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'op_features: clone-child' 2023-09-29T18:27:09.969 INFO:tasks.workunit.client.0.smithi143.stdout: op_features: clone-child 2023-09-29T18:27:09.969 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v1 - 2023-09-29T18:27:09.969 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:10.010 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:10.014 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:10.014 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v2 - 2023-09-29T18:27:10.014 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:10.055 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:10.058 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:10.059 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children test1@snap1 2023-09-29T18:27:10.101 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-09-29T18:27:10.101 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children test1@snap2 2023-09-29T18:27:10.137 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-09-29T18:27:10.137 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test1 rbd2/test2 2023-09-29T18:27:10.227 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:10.226+0000 7f0d94d96700 0 -- 172.21.15.143:0/1050022810 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f0d70008740 msgr2=0x7f0d7000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:11.678 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:11.677+0000 7f0d94d96700 0 -- 172.21.15.143:0/1050022810 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f0d7803aa00 msgr2=0x7f0d7803cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:11.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v1 2023-09-29T18:27:11.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd2/test2@snap1' 2023-09-29T18:27:11.797 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd2/test2@snap1 2023-09-29T18:27:11.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:11.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd2/test2@snap2' 2023-09-29T18:27:11.850 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd2/test2@snap2 2023-09-29T18:27:11.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:11.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'op_features: clone-child' 2023-09-29T18:27:11.903 INFO:tasks.workunit.client.0.smithi143.stdout: op_features: clone-child 2023-09-29T18:27:11.903 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children rbd2/test2@snap1 2023-09-29T18:27:11.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-09-29T18:27:11.956 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children rbd2/test2@snap2 2023-09-29T18:27:12.004 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-09-29T18:27:12.004 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:12.090 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:12.093 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd migration commit test1 2023-09-29T18:27:12.093 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:12.162 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: the image has descendants: 2023-09-29T18:27:12.162 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v1 2023-09-29T18:27:12.162 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v2 2023-09-29T18:27:12.162 INFO:tasks.workunit.client.0.smithi143.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-09-29T18:27:12.162 INFO:tasks.workunit.client.0.smithi143.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-09-29T18:27:12.164 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:12.165 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 --force 2023-09-29T18:27:12.232 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: the image has descendants: 2023-09-29T18:27:12.232 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v1 2023-09-29T18:27:12.233 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v2 2023-09-29T18:27:12.233 INFO:tasks.workunit.client.0.smithi143.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-09-29T18:27:12.233 INFO:tasks.workunit.client.0.smithi143.stderr:Proceeding anyway due to force flag set. 2023-09-29T18:27:12.244 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:12.244+0000 7f4b031f3700 0 -- 172.21.15.143:0/445021482 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5597f1a17100 msgr2=0x5597f1a194e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:12.251 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:12.250+0000 7f4b031f3700 0 -- 172.21.15.143:0/445021482 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f4ae803aa00 msgr2=0x7f4ae803cde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:13.617 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:13.621 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v1 - 2023-09-29T18:27:13.621 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:13.666 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:13.669 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:13.670 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v2 - 2023-09-29T18:27:13.670 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:13.710 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:13.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:13.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare rbd2/test2 test1 2023-09-29T18:27:13.816 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:13.816+0000 7f697c9b5700 0 -- 172.21.15.143:0/94951688 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f6958008740 msgr2=0x7f695800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:15.692 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:15.691+0000 7f697c9b5700 0 -- 172.21.15.143:0/94951688 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f696003ab40 msgr2=0x7f696003cf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:15.775 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v1 2023-09-29T18:27:15.775 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-09-29T18:27:15.830 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd/test1@snap1 2023-09-29T18:27:15.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:15.831 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-09-29T18:27:15.883 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd/test1@snap2 2023-09-29T18:27:15.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info clone_v2 2023-09-29T18:27:15.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ fgrep 'op_features: clone-child' 2023-09-29T18:27:15.936 INFO:tasks.workunit.client.0.smithi143.stdout: op_features: clone-child 2023-09-29T18:27:15.936 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children test1@snap1 2023-09-29T18:27:15.990 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-09-29T18:27:15.990 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd children test1@snap2 2023-09-29T18:27:16.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-09-29T18:27:16.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test1 2023-09-29T18:27:16.140 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:16.139+0000 7fac9d750700 0 -- 172.21.15.143:0/1210693830 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fac780088b0 msgr2=0x7fac7800ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:16.144 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:16.147 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd migration commit test1 2023-09-29T18:27:16.147 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 2023-09-29T18:27:16.225 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: the image has descendants: 2023-09-29T18:27:16.225 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v1 2023-09-29T18:27:16.225 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v2 2023-09-29T18:27:16.225 INFO:tasks.workunit.client.0.smithi143.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-09-29T18:27:16.225 INFO:tasks.workunit.client.0.smithi143.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-09-29T18:27:16.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:16.228 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration commit test1 --force 2023-09-29T18:27:16.306 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: the image has descendants: 2023-09-29T18:27:16.306 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v1 2023-09-29T18:27:16.306 INFO:tasks.workunit.client.0.smithi143.stderr: rbd/clone_v2 2023-09-29T18:27:16.306 INFO:tasks.workunit.client.0.smithi143.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-09-29T18:27:16.306 INFO:tasks.workunit.client.0.smithi143.stderr:Proceeding anyway due to force flag set. 2023-09-29T18:27:16.308 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:16.308+0000 7ffae5bb8700 0 -- 172.21.15.143:0/3853725444 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7ffacc008740 msgr2=0x7ffacc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:16.319 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:16.318+0000 7ffae5bb8700 0 -- 172.21.15.143:0/3853725444 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7ffac803aa60 msgr2=0x7ffac803ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:17.728 INFO:tasks.workunit.client.0.smithi143.stderr: Commit image migration: 100% complete...done. 2023-09-29T18:27:17.731 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v1 - 2023-09-29T18:27:17.731 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:17.772 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:17.776 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:17.776 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export clone_v2 - 2023-09-29T18:27:17.776 INFO:tasks.workunit.client.0.smithi143.stderr:++ md5sum 2023-09-29T18:27:17.818 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:17.821 INFO:tasks.workunit.client.0.smithi143.stderr:+ test '44e5c0e4725a93a9afb08a45922f4509 -' = '44e5c0e4725a93a9afb08a45922f4509 -' 2023-09-29T18:27:17.821 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd remove clone_v1 2023-09-29T18:27:17.934 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:17.934+0000 7fb0911d1700 0 -- 172.21.15.143:0/3524839533 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fb07403aa70 msgr2=0x7fb07403ce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:17.949 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:17.952 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd remove clone_v2 2023-09-29T18:27:18.051 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:18.051+0000 7f1e501e5700 0 -- 172.21.15.143:0/1954108682 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f1e2c008740 msgr2=0x7f1e2c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:18.061 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:18.063 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect test1@snap1 2023-09-29T18:27:18.106 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge test1 2023-09-29T18:27:19.724 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:27:19.738 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:19.849 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:19.849+0000 7f9339850700 0 -- 172.21.15.143:0/506110023 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f931c03abd0 msgr2=0x7f931c03cff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:19.861 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:19.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ for format in 1 2 2023-09-29T18:27:19.864 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 1 test2 2023-09-29T18:27:19.887 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:19.894 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:19.894+0000 7f7230ad6500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:19.900 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-09-29T18:27:19.983 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:20.025 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:20.033 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:20.033 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 125.001 bytes/sec: 125 KiB/s 2023-09-29T18:27:20.040 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:20.121 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:20.121+0000 7f3c62eb4700 0 -- 172.21.15.143:0/2230985116 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f3c400088b0 msgr2=0x7f3c4000ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:20.140 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:20.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:20.179 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:20.184 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:20.184 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 166.668 bytes/sec: 167 KiB/s 2023-09-29T18:27:20.189 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:20.249 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:20.252 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 1 test2 2023-09-29T18:27:20.275 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:20.281 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:20.280+0000 7f3889875500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:20.288 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-09-29T18:27:20.359 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:20.402 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:20.405 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:20.405 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 250.002 bytes/sec: 250 KiB/s 2023-09-29T18:27:20.410 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test2 2023-09-29T18:27:20.497 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:20.497+0000 7f6889257700 0 -- 172.21.15.143:0/283022148 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f6864008740 msgr2=0x7f686400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:20.521 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:20.520+0000 7f6889a58700 0 -- 172.21.15.143:0/283022148 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5608b8525930 msgr2=0x5608b8616770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:20.536 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:20.539 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:20.815 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:20.815+0000 7f502e4e5700 0 -- 172.21.15.143:0/3086728066 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f5008008740 msgr2=0x7f500800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:20.844 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:20.843+0000 7f502ece6700 0 -- 172.21.15.143:0/3086728066 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55f0073ee930 msgr2=0x55f0074dd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:20.847 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:20.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:20.886 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:20.889 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:20.889 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 333.336 bytes/sec: 333 KiB/s 2023-09-29T18:27:20.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:20.946 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:20.948 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 1 test2 2023-09-29T18:27:20.972 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:20.978 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:20.978+0000 7fc0f0db7500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:20.984 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-09-29T18:27:21.028 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:21.027+0000 7f9685572500 -1 librbd::image::CreateRequest: 0x5653eff5f190 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-09-29T18:27:21.028 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:21.027+0000 7f9685572500 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-09-29T18:27:21.035 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-09-29T18:27:21.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:21.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:21.073 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:21.078 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:21.078 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 200.002 bytes/sec: 200 KiB/s 2023-09-29T18:27:21.085 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:21.149 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:21.151 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 1 test2 2023-09-29T18:27:21.175 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:21.180 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:21.180+0000 7f865da23500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:21.188 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 rbd2/test2 2023-09-29T18:27:21.288 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-09-29T18:27:21.330 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:21.333 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:21.334 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 250.002 bytes/sec: 250 KiB/s 2023-09-29T18:27:21.342 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:21.433 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-09-29T18:27:21.436 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:21.472 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:21.474 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:21.474 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 333.336 bytes/sec: 333 KiB/s 2023-09-29T18:27:21.479 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:21.530 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:21.532 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 1 test2 2023-09-29T18:27:21.556 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image format 1 is deprecated 2023-09-29T18:27:21.562 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:21.561+0000 7fc3ea171500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:21.568 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 rbd2/test2 2023-09-29T18:27:21.640 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort rbd2/test2 2023-09-29T18:27:21.723 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:21.723+0000 7fc175008700 0 -- 172.21.15.143:0/2884529715 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fc150008740 msgr2=0x7fc15000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:21.741 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:21.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:21.780 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:21.782 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:21.783 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 333.336 bytes/sec: 333 KiB/s 2023-09-29T18:27:21.787 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:21.848 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:21.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ test 1 = 1 2023-09-29T18:27:21.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ continue 2023-09-29T18:27:21.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ for format in 1 2 2023-09-29T18:27:21.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:21.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-09-29T18:27:21.995 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:21.994+0000 7f8f4fe52700 0 -- 172.21.15.143:0/549709434 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5577830b2760 msgr2=0x557782f59600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:22.050 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:22.061 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:22.061 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 90.9099 bytes/sec: 91 KiB/s 2023-09-29T18:27:22.074 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:22.192 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:22.192+0000 7ff5a7da8700 0 -- 172.21.15.143:0/2565653016 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55d2b955bd90 msgr2=0x55d2b95b9670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.199 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:22.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:22.244 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:22.246 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:22.246 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 500.005 bytes/sec: 500 KiB/s 2023-09-29T18:27:22.253 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:22.349 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:22.348+0000 7fab75860700 0 -- 172.21.15.143:0/1818462867 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55cf3ea047b0 msgr2=0x55cf3eaaa5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.356 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:22.359 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:22.403 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-09-29T18:27:22.525 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:22.523+0000 7f870f5c4700 0 -- 172.21.15.143:0/84517941 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f86ec0088b0 msgr2=0x7f86ec00ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.531 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:22.582 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:22.597 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:22.598 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 66.6673 bytes/sec: 67 KiB/s 2023-09-29T18:27:22.609 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration execute test2 2023-09-29T18:27:22.705 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-09-29T18:27:22.704+0000 7f27550f0700 0 -- 172.21.15.143:0/657562908 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55c038a1a7b0 msgr2=0x55c038b0f730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.727 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:22.727+0000 7f27548ef700 0 -- 172.21.15.143:0/657562908 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f2730008740 msgr2=0x7f273000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:22.729 INFO:tasks.workunit.client.0.smithi143.stderr: Image migration: 100% complete...done. 2023-09-29T18:27:22.732 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:22.839 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-09-29T18:27:22.838+0000 7f7ff3ec5700 0 -- 172.21.15.143:0/1825611818 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7fd00088b0 msgr2=0x7f7fd000ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:22.841 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 100% complete...done. 2023-09-29T18:27:22.844 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:22.888 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:22.890 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:22.890 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 500.005 bytes/sec: 500 KiB/s 2023-09-29T18:27:22.899 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:23.007 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:23.007+0000 7f9939a3b700 0 -- 172.21.15.143:0/53277601 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f9920008740 msgr2=0x7f992000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:23.014 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:23.017 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:23.061 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-09-29T18:27:23.130 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:23.130+0000 7fb952231500 -1 librbd::image::CreateRequest: 0x561ee9c190e0 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-09-29T18:27:23.131 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:23.130+0000 7fb952231500 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-09-29T18:27:23.153 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-09-29T18:27:23.156 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:23.156 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:23.199 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:23.204 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:23.204 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 166.668 bytes/sec: 167 KiB/s 2023-09-29T18:27:23.218 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:23.328 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:23.327+0000 7f3750f0a700 0 -- 172.21.15.143:0/408709379 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55de836767b0 msgr2=0x55de8371c620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:23.336 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:23.342 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:23.383 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 rbd2/test2 2023-09-29T18:27:23.490 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:23.490+0000 7f47a2c5a700 0 -- 172.21.15.143:0/1206475281 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f47800064f0 msgr2=0x7f4780015cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:23.495 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-09-29T18:27:23.546 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:23.548 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:23.548 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 250.002 bytes/sec: 250 KiB/s 2023-09-29T18:27:23.555 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:23.646 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-09-29T18:27:23.649 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:23.692 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:23.694 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:23.694 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 500.005 bytes/sec: 500 KiB/s 2023-09-29T18:27:23.706 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:23.820 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:23.821+0000 7f0786e2b700 0 -- 172.21.15.143:0/1145296806 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f0768005fd0 msgr2=0x7f0768008450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:23.826 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:23.829 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:23.873 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 rbd2/test2 2023-09-29T18:27:23.968 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:23.968+0000 7f8120210700 0 -- 172.21.15.143:0/2442185875 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f80fc008740 msgr2=0x7f80fc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:23.978 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort rbd2/test2 2023-09-29T18:27:24.083 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-09-29T18:27:24.085 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:24.128 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:24.132 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:24.132 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 333.336 bytes/sec: 333 KiB/s 2023-09-29T18:27:24.141 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:24.237 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:24.238+0000 7fe746f5f700 0 -- 172.21.15.143:0/1045829263 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fe724008740 msgr2=0x7fe72400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:24.250 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:24.253 INFO:tasks.workunit.client.0.smithi143.stderr:+ test 2 = 1 2023-09-29T18:27:24.253 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 128M --image-format 2 test2 2023-09-29T18:27:24.296 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration prepare test2 rbd2/ns1/test3 2023-09-29T18:27:24.395 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:24.396+0000 7fa997dbc700 0 -- 172.21.15.143:0/1755142857 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55680d5b63a0 msgr2=0x55680d5b8780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:24.403 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/ns1/test3 2023-09-29T18:27:24.450 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:24.454 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:24.454 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 250.002 bytes/sec: 250 KiB/s 2023-09-29T18:27:24.465 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd migration abort test2 2023-09-29T18:27:24.582 INFO:tasks.workunit.client.0.smithi143.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-09-29T18:27:24.585 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-09-29T18:27:24.628 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-09-29T18:27:24.630 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:27:24.630 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 0 ops: 1 ops/sec: 500.004 bytes/sec: 500 KiB/s 2023-09-29T18:27:24.638 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:27:24.754 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:27:24.754+0000 7f78000d6700 0 -- 172.21.15.143:0/2457052615 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5604adeb5bd0 msgr2=0x5604adf56af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:24.764 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:24.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:24.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:24.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:24.931 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.016 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.094 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.171 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.249 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.332 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.409 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.490 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.571 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.649 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.727 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.805 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.881 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:25.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:26.035 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:26.112 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:26.192 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-09-29T18:27:26.845 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' does not exist 2023-09-29T18:27:26.854 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_config 2023-09-29T18:27:26.855 INFO:tasks.workunit.client.0.smithi143.stdout:testing config... 2023-09-29T18:27:26.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing config...' 2023-09-29T18:27:26.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:26.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:26.934 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.012 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.091 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.168 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.244 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.323 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.401 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.477 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.557 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.639 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.870 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:27.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:28.021 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:28.099 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:28.176 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:28.255 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global set osd rbd_cache true 2023-09-29T18:27:28.255 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set osd rbd_cache true 2023-09-29T18:27:28.278 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: invalid config entity: osd (must be global, client or client.) 2023-09-29T18:27:28.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global set global debug_ms 10 2023-09-29T18:27:28.279 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set global debug_ms 10 2023-09-29T18:27:28.302 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: not rbd option: debug_ms 2023-09-29T18:27:28.303 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.303 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global set global rbd_UNKNOWN false 2023-09-29T18:27:28.303 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set global rbd_UNKNOWN false 2023-09-29T18:27:28.327 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: invalid config key: rbd_UNKNOWN 2023-09-29T18:27:28.328 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.328 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global set global rbd_cache INVALID 2023-09-29T18:27:28.328 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set global rbd_cache INVALID 2023-09-29T18:27:28.356 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: error setting rbd_cache: error parsing value: Expected option value to be integer, got 'INVALID' 2023-09-29T18:27:28.358 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.358 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set global rbd_cache false 2023-09-29T18:27:28.391 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set client rbd_cache true 2023-09-29T18:27:28.424 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global set client.123 rbd_cache false 2023-09-29T18:27:28.457 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global get global rbd_cache 2023-09-29T18:27:28.457 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^false$' 2023-09-29T18:27:28.486 INFO:tasks.workunit.client.0.smithi143.stdout:false 2023-09-29T18:27:28.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global get client rbd_cache 2023-09-29T18:27:28.486 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^true$' 2023-09-29T18:27:28.515 INFO:tasks.workunit.client.0.smithi143.stdout:true 2023-09-29T18:27:28.515 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global get client.123 rbd_cache 2023-09-29T18:27:28.515 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^false$' 2023-09-29T18:27:28.544 INFO:tasks.workunit.client.0.smithi143.stdout:false 2023-09-29T18:27:28.544 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global get client.UNKNOWN rbd_cache 2023-09-29T18:27:28.544 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global get client.UNKNOWN rbd_cache 2023-09-29T18:27:28.571 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: rbd_cache is not set 2023-09-29T18:27:28.573 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.573 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global list global 2023-09-29T18:27:28.573 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * false * global *$' 2023-09-29T18:27:28.601 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache false global 2023-09-29T18:27:28.602 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global list client 2023-09-29T18:27:28.602 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * client *$' 2023-09-29T18:27:28.630 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true client 2023-09-29T18:27:28.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global list client.123 2023-09-29T18:27:28.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * false * client.123 *$' 2023-09-29T18:27:28.658 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache false client.123 2023-09-29T18:27:28.658 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global list client.UNKNOWN 2023-09-29T18:27:28.658 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * client *$' 2023-09-29T18:27:28.686 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true client 2023-09-29T18:27:28.686 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global rm client rbd_cache 2023-09-29T18:27:28.718 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config global get client rbd_cache 2023-09-29T18:27:28.718 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global get client rbd_cache 2023-09-29T18:27:28.745 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: rbd_cache is not set 2023-09-29T18:27:28.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:28.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global list client 2023-09-29T18:27:28.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * false * global *$' 2023-09-29T18:27:28.775 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache false global 2023-09-29T18:27:28.775 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global rm client.123 rbd_cache 2023-09-29T18:27:28.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config global rm global rbd_cache 2023-09-29T18:27:28.839 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool set rbd rbd_cache true 2023-09-29T18:27:28.879 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool list rbd 2023-09-29T18:27:28.879 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * pool *$' 2023-09-29T18:27:28.908 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true pool 2023-09-29T18:27:28.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool get rbd rbd_cache 2023-09-29T18:27:28.909 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^true$' 2023-09-29T18:27:28.938 INFO:tasks.workunit.client.0.smithi143.stdout:true 2023-09-29T18:27:28.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 1 test1 2023-09-29T18:27:28.965 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:28.965+0000 7f52c421d500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:28.973 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image list rbd/test1 2023-09-29T18:27:28.973 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * pool *$' 2023-09-29T18:27:29.011 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true pool 2023-09-29T18:27:29.011 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image set rbd/test1 rbd_cache false 2023-09-29T18:27:29.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image list rbd/test1 2023-09-29T18:27:29.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * false * image *$' 2023-09-29T18:27:29.092 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache false image 2023-09-29T18:27:29.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-09-29T18:27:29.092 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^false$' 2023-09-29T18:27:29.130 INFO:tasks.workunit.client.0.smithi143.stdout:false 2023-09-29T18:27:29.130 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image remove rbd/test1 rbd_cache 2023-09-29T18:27:29.172 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config image get rbd/test1 rbd_cache 2023-09-29T18:27:29.172 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-09-29T18:27:29.205 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: rbd_cache is not set 2023-09-29T18:27:29.210 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:29.210 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config image list rbd/test1 2023-09-29T18:27:29.210 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * pool *$' 2023-09-29T18:27:29.248 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true pool 2023-09-29T18:27:29.248 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool remove rbd rbd_cache 2023-09-29T18:27:29.285 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd config pool get rbd rbd_cache 2023-09-29T18:27:29.285 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool get rbd rbd_cache 2023-09-29T18:27:29.313 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: rbd_cache is not set 2023-09-29T18:27:29.315 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:27:29.315 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd config pool list rbd 2023-09-29T18:27:29.315 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^rbd_cache * true * config *$' 2023-09-29T18:27:29.344 INFO:tasks.workunit.client.0.smithi143.stdout:rbd_cache true config 2023-09-29T18:27:29.344 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:29.384 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:29.386 INFO:tasks.workunit.client.0.smithi143.stderr:+ RBD_CREATE_ARGS= 2023-09-29T18:27:29.386 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_others 2023-09-29T18:27:29.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-09-29T18:27:29.387 INFO:tasks.workunit.client.0.smithi143.stdout:testing import, export, resize, and snapshots... 2023-09-29T18:27:29.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-09-29T18:27:29.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:29.387 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.465 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.542 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.621 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.777 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:29.929 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.005 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.084 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.160 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.236 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.312 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.390 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.546 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.628 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.706 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:30.783 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-09-29T18:27:30.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-09-29T18:27:30.784 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records in 2023-09-29T18:27:30.785 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records out 2023-09-29T18:27:30.785 INFO:tasks.workunit.client.0.smithi143.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 3.993e-05 s, 25.6 MB/s 2023-09-29T18:27:30.785 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-09-29T18:27:30.785 INFO:tasks.workunit.client.0.smithi143.stderr:10+0 records in 2023-09-29T18:27:30.786 INFO:tasks.workunit.client.0.smithi143.stderr:10+0 records out 2023-09-29T18:27:30.786 INFO:tasks.workunit.client.0.smithi143.stderr:10240 bytes (10 kB, 10 KiB) copied, 5.0388e-05 s, 203 MB/s 2023-09-29T18:27:30.786 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-09-29T18:27:30.787 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records in 2023-09-29T18:27:30.787 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records out 2023-09-29T18:27:30.787 INFO:tasks.workunit.client.0.smithi143.stderr:72216 bytes (72 kB, 71 KiB) copied, 0.000175781 s, 411 MB/s 2023-09-29T18:27:30.787 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-09-29T18:27:30.789 INFO:tasks.workunit.client.0.smithi143.stderr:140+1 records in 2023-09-29T18:27:30.789 INFO:tasks.workunit.client.0.smithi143.stderr:140+1 records out 2023-09-29T18:27:30.789 INFO:tasks.workunit.client.0.smithi143.stderr:143368 bytes (143 kB, 140 KiB) copied, 0.000353913 s, 405 MB/s 2023-09-29T18:27:30.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-09-29T18:27:30.789 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records in 2023-09-29T18:27:30.790 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records out 2023-09-29T18:27:30.790 INFO:tasks.workunit.client.0.smithi143.stderr:72320 bytes (72 kB, 71 KiB) copied, 0.000183154 s, 395 MB/s 2023-09-29T18:27:30.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import /tmp/img1 testimg1 2023-09-29T18:27:30.815 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:30.815+0000 7f6dbe8ba500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:30.895 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-09-29T18:27:30.899 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-09-29T18:27:30.934 INFO:tasks.workunit.client.0.smithi143.stderr: Resizing image: 100% complete...done. 2023-09-29T18:27:30.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img2 2023-09-29T18:27:31.038 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:31.041 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1 --snap=snap1 2023-09-29T18:27:31.726 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:27:31.731 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=128 2023-09-29T18:27:31.765 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-09-29T18:27:31.770 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:27:31.770 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-09-29T18:27:31.817 INFO:tasks.workunit.client.0.smithi143.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-09-29T18:27:31.821 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img3 2023-09-29T18:27:31.887 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:31.890 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:27:31.890 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:31.922 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:31.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:27:31.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:31.954 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:31.955 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-09-29T18:27:31.955 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-09-29T18:27:32.048 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:32.052 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-09-29T18:27:32.111 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:27:32.115 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --size=1 testimg-diff1 2023-09-29T18:27:32.340 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:32.341+0000 7f1696862700 0 --2- 172.21.15.143:0/3505064649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ad26ff1bf0 0x55ad26feafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:27:32.343 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:32.344+0000 7f169ed50500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:32.353 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-09-29T18:27:32.584 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-09-29T18:27:32.589 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-09-29T18:27:32.649 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-09-29T18:27:32.654 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:27:32.654 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:32.688 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:32.689 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:27:32.689 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:32.721 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:32.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff1 2023-09-29T18:27:32.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:32.753 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:32.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-09-29T18:27:32.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:32.786 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:32.786 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-09-29T18:27:32.876 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:27:32.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg1 testimg3 2023-09-29T18:27:32.961 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:27:32.965 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-09-29T18:27:33.074 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:27:33.077 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-09-29T18:27:33.165 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:27:33.169 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg2 2023-09-29T18:27:33.169 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:33.204 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:33.204 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:27:33.204 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:33.237 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:33.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff2 2023-09-29T18:27:33.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:33.270 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:33.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff3 2023-09-29T18:27:33.271 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:33.304 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:33.304 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy testimg1 testimg4 2023-09-29T18:27:33.766 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete...2023-09-29T18:27:33.767+0000 7f7298919700 0 -- 172.21.15.143:0/1435044811 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f7278001be0 msgr2=0x7f7278004090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:33.776 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 100% complete...done. 2023-09-29T18:27:33.778 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-09-29T18:27:34.792 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete...2023-09-29T18:27:34.792+0000 7fe78dc81700 0 -- 172.21.15.143:0/825373869 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fe768001be0 msgr2=0x7fe768004090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:34.792 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 100% complete...done. 2023-09-29T18:27:34.796 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg4 2023-09-29T18:27:34.797 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:27:34.832 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:27:34.832 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg5 2023-09-29T18:27:34.833 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:34.866 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:34.867 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg4 2023-09-29T18:27:34.867 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:27:34.867 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:34.867 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:27:34.901 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:27:34.901 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg4 2023-09-29T18:27:34.901 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '.*snap1.*' 2023-09-29T18:27:34.936 INFO:tasks.workunit.client.0.smithi143.stdout: 18 snap1 256 MiB Fri Sep 29 18:27:33 2023 2023-09-29T18:27:34.936 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img1.new 2023-09-29T18:27:35.004 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.007 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg2 /tmp/img2.new 2023-09-29T18:27:35.117 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.119 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg3 /tmp/img3.new 2023-09-29T18:27:35.192 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.195 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-09-29T18:27:35.263 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.266 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-09-29T18:27:35.370 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.374 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-09-29T18:27:35.442 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:35.477 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-09-29T18:27:35.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-09-29T18:27:35.738 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-09-29T18:27:35.897 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-09-29T18:27:35.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-09-29T18:27:36.064 INFO:tasks.workunit.client.0.smithi143.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-09-29T18:27:36.068 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-09-29T18:27:36.165 INFO:tasks.workunit.client.0.smithi143.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-09-29T18:27:36.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:27:36.175 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:36.208 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:36.208 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff1 2023-09-29T18:27:36.209 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:27:36.241 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:27:36.241 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-09-29T18:27:36.336 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:36.339 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-09-29T18:27:36.441 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:27:36.444 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-09-29T18:27:36.595 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-09-29T18:27:36.743 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg2 2023-09-29T18:27:36.839 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete...2023-09-29T18:27:36.840+0000 7f2e947aa700 0 -- 172.21.15.143:0/2322479375 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55c1553aeca0 msgr2=0x55c155450b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:36.862 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-09-29T18:27:36.863+0000 7f2e93fa9700 0 -- 172.21.15.143:0/2322479375 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f2e70008740 msgr2=0x7f2e7000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:36.863 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:36.865 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg3 2023-09-29T18:27:36.955 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-09-29T18:27:36.957 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create testimg2 -s 0 2023-09-29T18:27:36.985 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:36.986+0000 7f6629e8c500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:36.992 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd cp testimg2 testimg3 2023-09-29T18:27:37.041 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete...done. 2023-09-29T18:27:37.044 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep cp testimg2 testimg6 2023-09-29T18:27:37.099 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 100% complete...done. 2023-09-29T18:27:37.101 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-09-29T18:27:37.738 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:27:37.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-09-29T18:27:38.741 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:27:38.748 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:27:38.748 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-09-29T18:27:38.786 INFO:tasks.workunit.client.0.smithi143.stdout:error setting snapshot context: (2) No such file or directory 2023-09-29T18:27:38.787 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-09-29T18:27:38.787 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-09-29T18:27:38.819 INFO:tasks.workunit.client.0.smithi143.stdout:error setting snapshot context: (2) No such file or directory 2023-09-29T18:27:38.820 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd sparsify testimg1 2023-09-29T18:27:38.922 INFO:tasks.workunit.client.0.smithi143.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-09-29T18:27:38.932 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:38.933 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:39.058 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:39.135 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:39.260 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:39.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:40.887 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.026 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.139 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.447 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.525 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.604 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.843 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:41.920 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.082 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.162 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-09-29T18:27:42.277 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_locking 2023-09-29T18:27:42.278 INFO:tasks.workunit.client.0.smithi143.stdout:testing locking... 2023-09-29T18:27:42.278 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing locking...' 2023-09-29T18:27:42.278 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:42.278 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.355 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.435 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.513 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.791 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:42.945 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.102 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.180 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.260 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.336 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.413 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.490 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.566 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.720 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:43.875 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create -s 1 test1 2023-09-29T18:27:43.902 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:43.902+0000 7fa50c7e5500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:43.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:43.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:43.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:27:43.941 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:27:43.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id 2023-09-29T18:27:43.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:43.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 1 ' 2023-09-29T18:27:44.012 INFO:tasks.workunit.client.0.smithi143.stdout:There is 1 exclusive lock on this image. 2023-09-29T18:27:44.013 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd lock list test1 2023-09-29T18:27:44.013 INFO:tasks.workunit.client.0.smithi143.stderr:++ tail -n 1 2023-09-29T18:27:44.013 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{print $1;}' 2023-09-29T18:27:44.045 INFO:tasks.workunit.client.0.smithi143.stderr:+ LOCKER=client.7792 2023-09-29T18:27:44.045 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock remove test1 id client.7792 2023-09-29T18:27:44.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:44.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:44.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:27:44.789 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:27:44.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id --shared tag 2023-09-29T18:27:44.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:44.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 1 ' 2023-09-29T18:27:44.861 INFO:tasks.workunit.client.0.smithi143.stdout:There is 1 shared lock on this image. 2023-09-29T18:27:44.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id --shared tag 2023-09-29T18:27:44.902 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:44.902 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 2 ' 2023-09-29T18:27:44.933 INFO:tasks.workunit.client.0.smithi143.stdout:There are 2 shared locks on this image. 2023-09-29T18:27:44.933 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id2 --shared tag 2023-09-29T18:27:44.974 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:44.975 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 3 ' 2023-09-29T18:27:45.006 INFO:tasks.workunit.client.0.smithi143.stdout:There are 3 shared locks on this image. 2023-09-29T18:27:45.006 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:27:45.006 INFO:tasks.workunit.client.0.smithi143.stderr:+ tail -n 1 2023-09-29T18:27:45.007 INFO:tasks.workunit.client.0.smithi143.stderr:+ awk '{print $2, $1;}' 2023-09-29T18:27:45.007 INFO:tasks.workunit.client.0.smithi143.stderr:+ xargs rbd lock remove test1 2023-09-29T18:27:45.765 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:27:45.765 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -qE 'features:.*exclusive' 2023-09-29T18:27:45.800 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:45.841 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:27:45.843 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_thick_provision 2023-09-29T18:27:45.843 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing thick provision...' 2023-09-29T18:27:45.844 INFO:tasks.workunit.client.0.smithi143.stdout:testing thick provision... 2023-09-29T18:27:45.844 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:45.844 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:45.920 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:45.996 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.073 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.149 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.226 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.302 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.379 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.456 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.534 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.612 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.689 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.848 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:46.925 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:47.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:47.076 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:47.153 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:47.230 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --thick-provision -s 64M test1 2023-09-29T18:27:47.258 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:47.258+0000 7fb645e8f500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:47.525 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-09-29T18:27:47.535 INFO:tasks.workunit.client.0.smithi143.stderr:+ count=0 2023-09-29T18:27:47.535 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret= 2023-09-29T18:27:47.535 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 -lt 10 ']' 2023-09-29T18:27:47.535 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:27:47.535 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:27:47.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr -s ' ' 2023-09-29T18:27:47.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ cut -d ' ' -f 4-5 2023-09-29T18:27:47.536 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^64 MiB' 2023-09-29T18:27:47.565 INFO:tasks.workunit.client.0.smithi143.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-09-29T18:27:47.570 INFO:tasks.workunit.client.0.smithi143.stdout:64 MiB 2023-09-29T18:27:47.571 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret=0 2023-09-29T18:27:47.571 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 = 0 ']' 2023-09-29T18:27:47.571 INFO:tasks.workunit.client.0.smithi143.stderr:+ break 2023-09-29T18:27:47.571 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:27:47.602 INFO:tasks.workunit.client.0.smithi143.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-09-29T18:27:47.606 INFO:tasks.workunit.client.0.smithi143.stdout:NAME PROVISIONED USED 2023-09-29T18:27:47.606 INFO:tasks.workunit.client.0.smithi143.stdout:test1 64 MiB 64 MiB 2023-09-29T18:27:47.609 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 '!=' 0 ']' 2023-09-29T18:27:47.609 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:47.671 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete... Removing image: 100% complete...done. 2023-09-29T18:27:47.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:27:47.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:27:47.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:47.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:27:47.900 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:47.901+0000 7f9f787ce700 0 --2- 172.21.15.143:0/1085614601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631e2085cc0 0x5631e2080c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:27:47.906 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:27:47.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --thick-provision -s 4G test1 2023-09-29T18:27:47.934 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:47.935+0000 7f362ce14500 -1 librbd: Forced V1 image creation. 2023-09-29T18:27:48.890 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete... Thick provisioning: 6% complete... Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete...2023-09-29T18:27:48.889+0000 7f3625127700 0 -- 172.21.15.143:0/2537631089 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55dce7b54380 msgr2=0x55dce7b470a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:49.087 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 12% complete...2023-09-29T18:27:49.087+0000 7f3624926700 0 -- 172.21.15.143:0/2537631089 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55dce7bf5170 msgr2=0x55dce7bf7550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:57.536 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 13% complete... Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ count=0 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret= 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 -lt 10 ']' 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr -s ' ' 2023-09-29T18:27:58.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ cut -d ' ' -f 4-5 2023-09-29T18:27:58.035 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^4 GiB' 2023-09-29T18:27:58.035 INFO:tasks.workunit.client.0.smithi143.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-09-29T18:27:58.035 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:57.591+0000 7fe487822700 0 -- 172.21.15.143:0/2175001470 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55fa827db9c0 msgr2=0x55fa827eb370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:58.035 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:57.594+0000 7fe487822700 0 -- 172.21.15.143:0/2175001470 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fe46c03a4e0 msgr2=0x7fe46c03c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:58.035 INFO:tasks.workunit.client.0.smithi143.stdout:4 GiB 2023-09-29T18:27:58.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret=0 2023-09-29T18:27:58.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 = 0 ']' 2023-09-29T18:27:58.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ break 2023-09-29T18:27:58.036 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:27:58.036 INFO:tasks.workunit.client.0.smithi143.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-09-29T18:27:58.037 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:57.779+0000 7f47ed9e0700 0 -- 172.21.15.143:0/4209776093 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x562d4c038dc0 msgr2=0x562d4c0c8ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:58.037 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:27:57.782+0000 7f47ed1df700 0 -- 172.21.15.143:0/4209776093 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x562d4c0e7950 msgr2=0x562d4c0e9d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:58.037 INFO:tasks.workunit.client.0.smithi143.stdout:NAME PROVISIONED USED 2023-09-29T18:27:58.037 INFO:tasks.workunit.client.0.smithi143.stdout:test1 4 GiB 4 GiB 2023-09-29T18:27:58.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 '!=' 0 ']' 2023-09-29T18:27:58.038 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:27:58.039 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete...2023-09-29T18:27:58.023+0000 7f9661002700 0 -- 172.21.15.143:0/3964570900 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f964403a490 msgr2=0x7f964403c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:27:58.041 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 11% complete... Removing image: 12% complete...2023-09-29T18:27:58.042+0000 7f9661002700 0 -- 172.21.15.143:0/3964570900 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55ed13751980 msgr2=0x7f964403cf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:27:58.863 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-09-29T18:27:58.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:27:58.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:27:58.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:27:58.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:27:58.892 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:27:58.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ RBD_CREATE_ARGS='--image-format 2' 2023-09-29T18:27:58.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_others 2023-09-29T18:27:58.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-09-29T18:27:58.894 INFO:tasks.workunit.client.0.smithi143.stdout:testing import, export, resize, and snapshots... 2023-09-29T18:27:58.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-09-29T18:27:58.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:27:58.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:58.972 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.051 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.127 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.206 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.283 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.361 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.439 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.517 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.594 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.674 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.752 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.907 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:27:59.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:00.062 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:00.139 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:00.217 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:00.294 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-09-29T18:28:00.295 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-09-29T18:28:00.296 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records in 2023-09-29T18:28:00.296 INFO:tasks.workunit.client.0.smithi143.stderr:1+0 records out 2023-09-29T18:28:00.296 INFO:tasks.workunit.client.0.smithi143.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.3923e-05 s, 23.3 MB/s 2023-09-29T18:28:00.296 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-09-29T18:28:00.297 INFO:tasks.workunit.client.0.smithi143.stderr:10+0 records in 2023-09-29T18:28:00.297 INFO:tasks.workunit.client.0.smithi143.stderr:10+0 records out 2023-09-29T18:28:00.297 INFO:tasks.workunit.client.0.smithi143.stderr:10240 bytes (10 kB, 10 KiB) copied, 5.0427e-05 s, 203 MB/s 2023-09-29T18:28:00.297 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-09-29T18:28:00.298 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records in 2023-09-29T18:28:00.298 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records out 2023-09-29T18:28:00.298 INFO:tasks.workunit.client.0.smithi143.stderr:72216 bytes (72 kB, 71 KiB) copied, 0.000166239 s, 434 MB/s 2023-09-29T18:28:00.298 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:140+1 records in 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:140+1 records out 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:143368 bytes (143 kB, 140 KiB) copied, 0.000305563 s, 469 MB/s 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records in 2023-09-29T18:28:00.299 INFO:tasks.workunit.client.0.smithi143.stderr:70+1 records out 2023-09-29T18:28:00.300 INFO:tasks.workunit.client.0.smithi143.stderr:72320 bytes (72 kB, 71 KiB) copied, 0.000173369 s, 417 MB/s 2023-09-29T18:28:00.300 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import --image-format 2 /tmp/img1 testimg1 2023-09-29T18:28:00.423 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-09-29T18:28:00.426 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-09-29T18:28:00.469 INFO:tasks.workunit.client.0.smithi143.stderr: Resizing image: 100% complete...done. 2023-09-29T18:28:00.477 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img2 2023-09-29T18:28:00.574 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:00.577 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1 --snap=snap1 2023-09-29T18:28:00.801 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:00.802+0000 7f1be5200700 0 --2- 172.21.15.143:0/2270590035 >> v2:172.21.15.143:3300/0 conn(0x55abab2f68a0 0x55abab2f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:28:01.691 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:01.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=128 2023-09-29T18:28:01.736 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-09-29T18:28:01.740 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:28:01.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-09-29T18:28:01.785 INFO:tasks.workunit.client.0.smithi143.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-09-29T18:28:01.794 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img3 2023-09-29T18:28:01.866 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:01.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:28:01.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:01.903 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:01.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:28:01.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:01.939 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:01.940 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-09-29T18:28:01.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-09-29T18:28:01.993 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:01.996 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-09-29T18:28:02.042 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 100% complete...done. 2023-09-29T18:28:02.045 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size=1 testimg-diff1 2023-09-29T18:28:02.090 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-09-29T18:28:02.693 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-09-29T18:28:02.701 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-09-29T18:28:02.747 INFO:tasks.workunit.client.0.smithi143.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-09-29T18:28:02.756 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:28:02.756 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:02.790 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:02.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:28:02.790 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:02.825 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:02.826 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff1 2023-09-29T18:28:02.826 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:02.859 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:02.860 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-09-29T18:28:02.860 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:02.896 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:02.897 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-09-29T18:28:03.000 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:28:03.002 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg1 testimg3 2023-09-29T18:28:03.106 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:28:03.108 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-09-29T18:28:03.217 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:28:03.220 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-09-29T18:28:03.340 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-09-29T18:28:03.343 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg2 2023-09-29T18:28:03.343 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:03.376 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:03.377 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:28:03.377 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:03.410 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:03.410 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff2 2023-09-29T18:28:03.410 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:03.443 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:03.444 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff3 2023-09-29T18:28:03.444 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:03.476 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:03.477 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy testimg1 testimg4 2023-09-29T18:28:03.742 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-09-29T18:28:03.745 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-09-29T18:28:04.766 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-09-29T18:28:04.770 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg4 2023-09-29T18:28:04.770 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 128 MiB' 2023-09-29T18:28:04.803 INFO:tasks.workunit.client.0.smithi143.stdout: size 128 MiB in 32 objects 2023-09-29T18:28:04.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg5 2023-09-29T18:28:04.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:04.837 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:04.837 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg4 2023-09-29T18:28:04.837 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:28:04.837 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:04.838 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:04.870 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:04.871 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg4 2023-09-29T18:28:04.871 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '.*snap1.*' 2023-09-29T18:28:04.904 INFO:tasks.workunit.client.0.smithi143.stdout: 26 snap1 256 MiB Fri Sep 29 18:28:03 2023 2023-09-29T18:28:04.905 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img1.new 2023-09-29T18:28:04.975 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:04.978 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg2 /tmp/img2.new 2023-09-29T18:28:05.072 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:05.075 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg3 /tmp/img3.new 2023-09-29T18:28:05.146 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:05.150 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-09-29T18:28:05.225 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:05.227 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-09-29T18:28:05.332 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:05.336 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-09-29T18:28:05.405 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:05.408 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-09-29T18:28:05.604 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-09-29T18:28:05.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-09-29T18:28:05.858 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-09-29T18:28:05.936 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-09-29T18:28:06.034 INFO:tasks.workunit.client.0.smithi143.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-09-29T18:28:06.043 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-09-29T18:28:06.133 INFO:tasks.workunit.client.0.smithi143.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-09-29T18:28:06.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg1 2023-09-29T18:28:06.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:06.182 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:06.183 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg-diff1 2023-09-29T18:28:06.183 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:28:06.216 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:28:06.216 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-09-29T18:28:06.319 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:06.322 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-09-29T18:28:06.424 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-09-29T18:28:06.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-09-29T18:28:06.578 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-09-29T18:28:06.727 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg2 2023-09-29T18:28:06.832 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-09-29T18:28:06.833+0000 7f1f0fe5f700 0 -- 172.21.15.143:0/1257568502 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55e4bfb17280 msgr2=0x55e4bfb28c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:06.843 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:06.845 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg3 2023-09-29T18:28:07.154 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-09-29T18:28:07.155+0000 7f66f8b2a700 0 -- 172.21.15.143:0/830602486 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x556eb5513280 msgr2=0x556eb5524740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:07.173 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:07.176 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create testimg2 -s 0 2023-09-29T18:28:07.203 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:07.204+0000 7f4a4cc00500 -1 librbd: Forced V1 image creation. 2023-09-29T18:28:07.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd cp testimg2 testimg3 2023-09-29T18:28:07.433 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:07.434+0000 7faa993dc700 0 --2- 172.21.15.143:0/1538445985 >> v2:172.21.15.143:3300/0 conn(0x562e061f5670 0x562e06286b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:28:07.464 INFO:tasks.workunit.client.0.smithi143.stderr: Image copy: 100% complete...done. 2023-09-29T18:28:07.467 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep cp testimg2 testimg6 2023-09-29T18:28:07.531 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 100% complete...done. 2023-09-29T18:28:07.534 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-09-29T18:28:07.697 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:07.714 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-09-29T18:28:08.705 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:08.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg1 2023-09-29T18:28:08.722 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-09-29T18:28:08.757 INFO:tasks.workunit.client.0.smithi143.stdout:error setting snapshot context: (2) No such file or directory 2023-09-29T18:28:08.757 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-09-29T18:28:08.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-09-29T18:28:08.791 INFO:tasks.workunit.client.0.smithi143.stdout:error setting snapshot context: (2) No such file or directory 2023-09-29T18:28:08.792 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd sparsify testimg1 2023-09-29T18:28:08.850 INFO:tasks.workunit.client.0.smithi143.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-09-29T18:28:08.858 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:08.858 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:09.023 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:09.099 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:09.222 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:09.842 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:10.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:10.960 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.114 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.276 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.437 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.518 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.599 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.679 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.762 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:11.920 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.000 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.284 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-09-29T18:28:12.477 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_locking 2023-09-29T18:28:12.478 INFO:tasks.workunit.client.0.smithi143.stdout:testing locking... 2023-09-29T18:28:12.478 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing locking...' 2023-09-29T18:28:12.478 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:12.478 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.561 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.721 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.801 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:12.958 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.037 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.195 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.274 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.352 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.430 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.509 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.590 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.669 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.747 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.826 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:13.906 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:28:13.952 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:13.953 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:13.953 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:28:13.989 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:13.989 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id 2023-09-29T18:28:14.030 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.030 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 1 ' 2023-09-29T18:28:14.064 INFO:tasks.workunit.client.0.smithi143.stdout:There is 1 exclusive lock on this image. 2023-09-29T18:28:14.065 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd lock list test1 2023-09-29T18:28:14.065 INFO:tasks.workunit.client.0.smithi143.stderr:++ tail -n 1 2023-09-29T18:28:14.065 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{print $1;}' 2023-09-29T18:28:14.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ LOCKER=client.8464 2023-09-29T18:28:14.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock remove test1 id client.8464 2023-09-29T18:28:14.718 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.719 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:14.719 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:28:14.755 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:14.755 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id --shared tag 2023-09-29T18:28:14.799 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.799 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 1 ' 2023-09-29T18:28:14.833 INFO:tasks.workunit.client.0.smithi143.stdout:There is 1 shared lock on this image. 2023-09-29T18:28:14.834 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id --shared tag 2023-09-29T18:28:14.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 2 ' 2023-09-29T18:28:14.910 INFO:tasks.workunit.client.0.smithi143.stdout:There are 2 shared locks on this image. 2023-09-29T18:28:14.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock add test1 id2 --shared tag 2023-09-29T18:28:14.954 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.954 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep ' 3 ' 2023-09-29T18:28:14.987 INFO:tasks.workunit.client.0.smithi143.stdout:There are 3 shared locks on this image. 2023-09-29T18:28:14.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:14.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ tail -n 1 2023-09-29T18:28:14.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ awk '{print $2, $1;}' 2023-09-29T18:28:14.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ xargs rbd lock remove test1 2023-09-29T18:28:15.726 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info test1 2023-09-29T18:28:15.727 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -qE 'features:.*exclusive' 2023-09-29T18:28:15.765 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd lock list test1 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' -n 'There are 2 shared locks on this image. 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:Lock tag: tag 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:Locker ID Address 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:client.8479 id 172.21.15.143:0/3508791666 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:client.8485 id 172.21.15.143:0/1518219910' ']' 2023-09-29T18:28:15.802 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:15.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ tail -n 1 2023-09-29T18:28:15.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ awk '{print $2, $1;}' 2023-09-29T18:28:15.803 INFO:tasks.workunit.client.0.smithi143.stderr:+ xargs rbd lock remove test1 2023-09-29T18:28:16.727 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd lock list test1 2023-09-29T18:28:16.762 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' -n 'There is 1 shared lock on this image. 2023-09-29T18:28:16.763 INFO:tasks.workunit.client.0.smithi143.stderr:Lock tag: tag 2023-09-29T18:28:16.763 INFO:tasks.workunit.client.0.smithi143.stderr:Locker ID Address 2023-09-29T18:28:16.763 INFO:tasks.workunit.client.0.smithi143.stderr:client.8479 id 172.21.15.143:0/3508791666' ']' 2023-09-29T18:28:16.763 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd lock list test1 2023-09-29T18:28:16.763 INFO:tasks.workunit.client.0.smithi143.stderr:+ tail -n 1 2023-09-29T18:28:16.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ awk '{print $2, $1;}' 2023-09-29T18:28:16.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ xargs rbd lock remove test1 2023-09-29T18:28:17.603 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd lock list test1 2023-09-29T18:28:17.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' -n '' ']' 2023-09-29T18:28:17.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:28:17.750 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:17.749+0000 7fa1badd7700 0 -- 172.21.15.143:0/3848250509 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fa198008740 msgr2=0x7fa19800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:17.750 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:17.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_clone 2023-09-29T18:28:17.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing clone...' 2023-09-29T18:28:17.754 INFO:tasks.workunit.client.0.smithi143.stdout:testing clone... 2023-09-29T18:28:17.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:17.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:17.833 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:17.911 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:17.990 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.068 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.147 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.225 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.304 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.382 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.461 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.819 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:18.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:19.055 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:19.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:19.212 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:19.291 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:19.370 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create test1 --image-format 2 -s 1 2023-09-29T18:28:19.415 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@s1 2023-09-29T18:28:19.737 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:19.746 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect test1@s1 2023-09-29T18:28:19.788 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool create rbd2 8 2023-09-29T18:28:20.853 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' already exists 2023-09-29T18:28:20.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd pool init rbd2 2023-09-29T18:28:23.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test1@s1 rbd2/clone 2023-09-29T18:28:23.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls 2023-09-29T18:28:23.825 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep clone 2023-09-29T18:28:23.854 INFO:tasks.workunit.client.0.smithi143.stdout:clone 2023-09-29T18:28:23.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls -l 2023-09-29T18:28:23.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep clone 2023-09-29T18:28:23.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1@s1 2023-09-29T18:28:23.897 INFO:tasks.workunit.client.0.smithi143.stdout:clone 1 MiB rbd/test1@s1 2 2023-09-29T18:28:23.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:23.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v clone 2023-09-29T18:28:23.925 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:28:23.925 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd flatten rbd2/clone 2023-09-29T18:28:23.968 INFO:tasks.workunit.client.0.smithi143.stderr: Image flatten: 100% complete...done. 2023-09-29T18:28:23.975 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create rbd2/clone@s1 2023-09-29T18:28:24.756 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:24.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect rbd2/clone@s1 2023-09-29T18:28:24.807 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone rbd2/clone@s1 clone2 2023-09-29T18:28:24.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:24.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep clone2 2023-09-29T18:28:24.907 INFO:tasks.workunit.client.0.smithi143.stdout:clone2 2023-09-29T18:28:24.907 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:28:24.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep clone2 2023-09-29T18:28:24.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep rbd2/clone@s1 2023-09-29T18:28:24.949 INFO:tasks.workunit.client.0.smithi143.stdout:clone2 1 MiB rbd2/clone@s1 2 2023-09-29T18:28:24.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd -p rbd2 ls 2023-09-29T18:28:24.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v clone2 2023-09-29T18:28:24.978 INFO:tasks.workunit.client.0.smithi143.stdout:clone 2023-09-29T18:28:24.978 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm clone2 2023-09-29T18:28:25.085 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:25.085+0000 7f86affe4700 0 -- 172.21.15.143:0/2558168322 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f868c008740 msgr2=0x7f868c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:28:25.088 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:25.091 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect rbd2/clone@s1 2023-09-29T18:28:25.334 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm rbd2/clone@s1 2023-09-29T18:28:25.800 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:25.886 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm rbd2/clone 2023-09-29T18:28:25.976 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:25.975+0000 7f0871b0f700 0 -- 172.21.15.143:0/1313469830 >> [v2:172.21.15.143:6816/86759,v1:172.21.15.143:6817/86759] conn(0x560531abeab0 msgr2=0x560531ac0f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:25.980 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:25.983 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect test1@s1 2023-09-29T18:28:26.028 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm test1@s1 2023-09-29T18:28:26.764 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:26.777 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:28:26.888 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:26.887+0000 7f0d131dd700 0 -- 172.21.15.143:0/750269163 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55873e1f2e80 msgr2=0x55873e1ca7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:26.889 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:26.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-09-29T18:28:27.881 INFO:tasks.workunit.client.0.smithi143.stderr:pool 'rbd2' does not exist 2023-09-29T18:28:27.890 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_trash 2023-09-29T18:28:27.890 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing trash...' 2023-09-29T18:28:27.891 INFO:tasks.workunit.client.0.smithi143.stdout:testing trash... 2023-09-29T18:28:27.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:27.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:27.974 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.053 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.132 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.290 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.369 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.448 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.527 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.606 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.685 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.765 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.843 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:28.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:29.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:29.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:29.158 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:29.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:29.318 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:28:29.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test2 2023-09-29T18:28:29.407 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:29.408 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:28:29.434 INFO:tasks.workunit.client.0.smithi143.stdout:test1 2023-09-29T18:28:29.435 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:29.435 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:28:29.461 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:28:29.462 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:29.462 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:29.462 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:29.488 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:29.489 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:28:29.489 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*2.*' 2023-09-29T18:28:29.525 INFO:tasks.workunit.client.0.smithi143.stdout:test1 1 MiB 2 2023-09-29T18:28:29.525 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:28:29.525 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*2.*' 2023-09-29T18:28:29.561 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 2 2023-09-29T18:28:29.562 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv test1 2023-09-29T18:28:29.621 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:29.621 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:28:29.648 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:28:29.648 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:29.648 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:29.648 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:29.675 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:29.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:28:29.675 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*2.*' 2023-09-29T18:28:29.710 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 2 2023-09-29T18:28:29.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:29.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:28:29.737 INFO:tasks.workunit.client.0.smithi143.stdout:226fb9b1e63d test1 2023-09-29T18:28:29.737 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:29.737 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:29.738 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:29.764 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:29.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:29.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*USER.*' 2023-09-29T18:28:29.799 INFO:tasks.workunit.client.0.smithi143.stdout:226fb9b1e63d test1 USER Fri Sep 29 18:28:29 2023 expired at Fri Sep 29 18:28:29 2023 2023-09-29T18:28:29.799 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:29.800 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v 'protected until' 2023-09-29T18:28:29.834 INFO:tasks.workunit.client.0.smithi143.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-09-29T18:28:29.834 INFO:tasks.workunit.client.0.smithi143.stdout:226fb9b1e63d test1 USER Fri Sep 29 18:28:29 2023 expired at Fri Sep 29 18:28:29 2023 2023-09-29T18:28:29.835 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:29.835 INFO:tasks.workunit.client.0.smithi143.stderr:++ cut -d ' ' -f 1 2023-09-29T18:28:29.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=226fb9b1e63d 2023-09-29T18:28:29.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm 226fb9b1e63d 2023-09-29T18:28:29.925 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:29.927 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv test2 2023-09-29T18:28:29.989 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:29.989 INFO:tasks.workunit.client.0.smithi143.stderr:++ cut -d ' ' -f 1 2023-09-29T18:28:30.018 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=22729a1792c2 2023-09-29T18:28:30.018 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info --image-id 22729a1792c2 2023-09-29T18:28:30.018 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'rbd image '\''test2'\''' 2023-09-29T18:28:30.050 INFO:tasks.workunit.client.0.smithi143.stdout:rbd image 'test2': 2023-09-29T18:28:30.050 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash restore 22729a1792c2 2023-09-29T18:28:30.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:30.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:28:30.115 INFO:tasks.workunit.client.0.smithi143.stdout:test2 2023-09-29T18:28:30.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:28:30.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:30.116 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:30.142 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:30.143 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls -l 2023-09-29T18:28:30.143 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*2.*' 2023-09-29T18:28:30.176 INFO:tasks.workunit.client.0.smithi143.stdout:test2 1 MiB 2 2023-09-29T18:28:30.177 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv test2 --expires-at '3600 sec' 2023-09-29T18:28:30.233 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image test2 will expire at 2023-09-29T19:28:30.206039+0000 2023-09-29T18:28:30.236 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:30.236 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test2 2023-09-29T18:28:30.262 INFO:tasks.workunit.client.0.smithi143.stdout:22729a1792c2 test2 2023-09-29T18:28:30.263 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:30.263 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:30.263 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:30.289 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:30.290 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:30.290 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test2.*USER.*protected until' 2023-09-29T18:28:30.324 INFO:tasks.workunit.client.0.smithi143.stdout:22729a1792c2 test2 USER Fri Sep 29 18:28:30 2023 protected until Fri Sep 29 19:28:30 2023 2023-09-29T18:28:30.324 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm 22729a1792c2 2023-09-29T18:28:30.324 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'Deferment time has not expired' 2023-09-29T18:28:30.352 INFO:tasks.workunit.client.0.smithi143.stdout:Deferment time has not expired, please use --force if you really want to remove the image 2023-09-29T18:28:30.352 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm --image-id 22729a1792c2 --force 2023-09-29T18:28:30.413 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:30.416 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:28:30.461 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@snap1 2023-09-29T18:28:30.774 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:30.783 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect test1@snap1 2023-09-29T18:28:30.824 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv test1 2023-09-29T18:28:30.883 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:30.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:28:30.910 INFO:tasks.workunit.client.0.smithi143.stdout:22c9ea64bf46 test1 2023-09-29T18:28:30.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:30.911 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:30.911 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:30.937 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:30.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:30.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*USER.*' 2023-09-29T18:28:30.971 INFO:tasks.workunit.client.0.smithi143.stdout:22c9ea64bf46 test1 USER Fri Sep 29 18:28:30 2023 expired at Fri Sep 29 18:28:30 2023 2023-09-29T18:28:30.972 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:30.972 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v 'protected until' 2023-09-29T18:28:31.006 INFO:tasks.workunit.client.0.smithi143.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-09-29T18:28:31.006 INFO:tasks.workunit.client.0.smithi143.stdout:22c9ea64bf46 test1 USER Fri Sep 29 18:28:30 2023 expired at Fri Sep 29 18:28:30 2023 2023-09-29T18:28:31.007 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:31.007 INFO:tasks.workunit.client.0.smithi143.stderr:++ cut -d ' ' -f 1 2023-09-29T18:28:31.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=22c9ea64bf46 2023-09-29T18:28:31.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls --image-id 22c9ea64bf46 2023-09-29T18:28:31.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:28:31.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:31.034 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:31.069 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:31.069 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls --image-id 22c9ea64bf46 2023-09-29T18:28:31.069 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '.*snap1.*' 2023-09-29T18:28:31.104 INFO:tasks.workunit.client.0.smithi143.stdout: 34 snap1 1 MiB yes Fri Sep 29 18:28:30 2023 2023-09-29T18:28:31.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect --image-id 22c9ea64bf46 --snap snap1 2023-09-29T18:28:31.353 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm --image-id 22c9ea64bf46 --snap snap1 2023-09-29T18:28:31.774 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:31.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls --image-id 22c9ea64bf46 2023-09-29T18:28:31.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:28:31.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:31.784 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:31.824 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:31.824 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash restore 22c9ea64bf46 2023-09-29T18:28:31.862 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@snap1 2023-09-29T18:28:32.613 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:32.623 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@snap2 2023-09-29T18:28:33.618 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:33.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls --image-id 22c9ea64bf46 2023-09-29T18:28:33.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:28:33.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:33.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:33.666 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:33.667 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 22c9ea64bf46 2023-09-29T18:28:35.780 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:28:35.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls --image-id 22c9ea64bf46 2023-09-29T18:28:35.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:28:35.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:35.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:35.830 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:35.831 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_to_trash_on_remove=true --rbd_move_to_trash_on_remove_expire_seconds=3600 test1 2023-09-29T18:28:35.891 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:35.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:35.894 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:28:35.921 INFO:tasks.workunit.client.0.smithi143.stdout:22c9ea64bf46 test1 2023-09-29T18:28:35.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:35.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:35.922 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:35.949 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:35.949 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -l 2023-09-29T18:28:35.949 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'test1.*USER.*protected until' 2023-09-29T18:28:35.983 INFO:tasks.workunit.client.0.smithi143.stdout:22c9ea64bf46 test1 USER Fri Sep 29 18:28:35 2023 protected until Fri Sep 29 19:28:35 2023 2023-09-29T18:28:35.984 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm 22c9ea64bf46 2023-09-29T18:28:35.984 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'Deferment time has not expired' 2023-09-29T18:28:36.011 INFO:tasks.workunit.client.0.smithi143.stdout:Deferment time has not expired, please use --force if you really want to remove the image2023-09-29T18:28:36.008+0000 7fb626fd2500 -1 librbd::api::Trash: remove: error: deferment time has not expired. 2023-09-29T18:28:36.011 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash rm --image-id 22c9ea64bf46 --force 2023-09-29T18:28:36.075 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:28:36.078 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:36.078 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.158 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.237 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.396 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.475 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.554 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.633 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.791 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.871 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:36.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.029 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.110 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.190 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.269 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.350 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.432 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.513 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_purge 2023-09-29T18:28:37.513 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing trash purge...' 2023-09-29T18:28:37.514 INFO:tasks.workunit.client.0.smithi143.stdout:testing trash purge... 2023-09-29T18:28:37.514 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:28:37.514 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.673 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.751 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.830 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.909 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:37.988 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.068 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.147 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.226 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.306 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.385 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.464 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.543 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.622 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.701 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.781 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.861 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:28:38.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:38.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:38.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:38.968 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:38.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:38.993 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete...done. 2023-09-29T18:28:38.995 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:39.041 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-09-29T18:28:39.088 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:39.151 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:39.212 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.212 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.212 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:39.238 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:39.238 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:39.334 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 50% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:39.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:39.364 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:39.364 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:39.409 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-09-29T18:28:39.456 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 --expires-at '1 hour' 2023-09-29T18:28:39.518 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image testimg1 will expire at 2023-09-29T19:28:39.486084+0000 2023-09-29T18:28:39.521 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 --expires-at '3 hours' 2023-09-29T18:28:39.584 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image testimg2 will expire at 2023-09-29T21:28:39.551059+0000 2023-09-29T18:28:39.587 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.587 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.587 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:39.614 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:39.614 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:39.638 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete...done. 2023-09-29T18:28:39.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.641 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:39.667 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:39.668 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge --expired-before 'now + 2 hours' 2023-09-29T18:28:39.732 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:39.735 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.735 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.735 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:39.762 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:39.762 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.762 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:28:39.789 INFO:tasks.workunit.client.0.smithi143.stdout:240d59c3c1b5 testimg2 2023-09-29T18:28:39.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge --expired-before 'now + 4 hours' 2023-09-29T18:28:39.854 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:39.857 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:39.857 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:39.857 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:39.883 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:39.884 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:39.928 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1@snap 2023-09-29T18:28:40.786 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:40.796 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-09-29T18:28:40.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:28:40.886 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:40.943 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:41.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:28:41.062 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:41.062 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:41.062 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:41.088 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:41.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:41.089 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:41.229 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:41.229 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:41.229 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:41.229 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:41.256 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:41.256 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:41.257 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:28:41.283 INFO:tasks.workunit.client.0.smithi143.stdout:242de52ac64d testimg1 2023-09-29T18:28:41.283 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:41.284 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{ print $1 }' 2023-09-29T18:28:41.311 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=242de52ac64d 2023-09-29T18:28:41.311 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 242de52ac64d 2023-09-29T18:28:41.793 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:28:41.802 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:41.863 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:41.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:41.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:41.866 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:41.892 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:41.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:41.939 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-09-29T18:28:41.986 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap 2023-09-29T18:28:42.610 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:42.623 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:28:42.671 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:42.730 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:42.789 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:28:42.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:42.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:42.851 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:42.877 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:42.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:42.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:43.019 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:43.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:43.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:43.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:43.046 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:43.047 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:43.047 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:28:43.073 INFO:tasks.workunit.client.0.smithi143.stdout:245d3cb812be testimg2 2023-09-29T18:28:43.073 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:43.074 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{ print $1 }' 2023-09-29T18:28:43.100 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=245d3cb812be 2023-09-29T18:28:43.101 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 245d3cb812be 2023-09-29T18:28:43.793 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:28:43.802 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:43.866 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:43.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:43.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:43.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:43.896 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:43.896 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:43.941 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-09-29T18:28:43.985 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:28:44.030 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg3@snap 2023-09-29T18:28:44.798 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:44.806 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:44.869 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:44.931 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:28:44.992 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:44.993 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:44.993 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:45.019 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:45.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:45.020 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:45.170 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:45.171 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:45.171 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:45.172 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:28:45.196 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:28:45.196 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:45.196 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg3 2023-09-29T18:28:45.223 INFO:tasks.workunit.client.0.smithi143.stdout:248dc6813d6f testimg3 2023-09-29T18:28:45.223 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:28:45.224 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{ print $1 }' 2023-09-29T18:28:45.250 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=248dc6813d6f 2023-09-29T18:28:45.251 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 248dc6813d6f 2023-09-29T18:28:45.795 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:28:45.804 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:45.871 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:45.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:45.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:45.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:45.901 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:45.901 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:45.946 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1@snap 2023-09-29T18:28:46.810 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:46.817 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-09-29T18:28:46.893 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg1@snap 2023-09-29T18:28:46.935 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:46.945 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:28:46.995 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap 2023-09-29T18:28:47.809 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:47.819 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-09-29T18:28:47.897 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-09-29T18:28:47.970 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg2@snap 2023-09-29T18:28:48.020 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:48.029 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg4@snap 2023-09-29T18:28:48.811 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:48.821 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-09-29T18:28:48.902 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg4@snap 2023-09-29T18:28:48.956 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:48.966 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:49.026 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:49.086 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:28:49.150 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg4 2023-09-29T18:28:49.213 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.213 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:49.213 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:28:49.239 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:28:49.239 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:49.240 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:49.463 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:49.464 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.464 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:49.464 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:49.491 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:49.491 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.491 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:28:49.518 INFO:tasks.workunit.client.0.smithi143.stdout:24b4eaa46656 testimg1 2023-09-29T18:28:49.518 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.518 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:28:49.544 INFO:tasks.workunit.client.0.smithi143.stdout:24ba774cef82 testimg2 2023-09-29T18:28:49.545 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.545 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg4 2023-09-29T18:28:49.572 INFO:tasks.workunit.client.0.smithi143.stdout:24c66eec355e testimg4 2023-09-29T18:28:49.572 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg6 2023-09-29T18:28:49.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:49.643 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:49.643 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:28:49.669 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:28:49.670 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:49.670 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:51.004 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:51.005 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:51.005 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:51.005 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:28:51.031 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:28:51.031 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:51.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:28:51.058 INFO:tasks.workunit.client.0.smithi143.stdout:24b4eaa46656 testimg1 2023-09-29T18:28:51.059 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:51.059 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:28:51.085 INFO:tasks.workunit.client.0.smithi143.stdout:24ba774cef82 testimg2 2023-09-29T18:28:51.086 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg5 2023-09-29T18:28:51.149 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:51.149 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:51.149 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:51.175 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:51.176 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:51.270 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:51.270+0000 7fa3227f5700 0 -- 172.21.15.143:0/3375606166 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55be20b2d3c0 msgr2=0x55be20b3ed90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:51.281 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:28:51.280+0000 7fa321ff4700 0 -- 172.21.15.143:0/3375606166 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fa2fc054af0 msgr2=0x7fa2fc056ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:28:52.673 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:28:52.676 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:52.676 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:52.676 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:28:52.703 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:28:52.703 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:28:52.748 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1@snap 2023-09-29T18:28:53.619 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:53.632 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-09-29T18:28:53.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg1@snap 2023-09-29T18:28:53.752 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:53.761 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:28:53.805 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg3@snap 2023-09-29T18:28:54.826 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:54.835 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap 2023-09-29T18:28:55.828 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:55.840 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-09-29T18:28:55.926 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-09-29T18:28:56.009 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg2@snap 2023-09-29T18:28:56.059 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:56.069 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg4@snap 2023-09-29T18:28:56.828 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:28:56.844 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-09-29T18:28:56.952 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg4@snap 2023-09-29T18:28:57.007 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:28:57.017 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg1 2023-09-29T18:28:57.079 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg2 2023-09-29T18:28:57.141 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:28:57.201 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg4 2023-09-29T18:28:57.263 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:57.264 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:57.264 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:28:57.290 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:28:57.291 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:57.291 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:57.426 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:57.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:57.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:57.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:28:57.454 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:28:57.454 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg6 2023-09-29T18:28:57.628 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:57.628 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:57.628 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 5 2023-09-29T18:28:57.654 INFO:tasks.workunit.client.0.smithi143.stdout:5 2023-09-29T18:28:57.655 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:57.655 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:28:59.054 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:28:59.055 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:59.055 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:59.055 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:28:59.081 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:28:59.082 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:59.082 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:28:59.309 INFO:tasks.workunit.client.0.smithi143.stdout:251476a217a0 testimg1 2023-09-29T18:28:59.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:59.310 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:28:59.336 INFO:tasks.workunit.client.0.smithi143.stdout:251a9dbdddc1 testimg2 2023-09-29T18:28:59.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:59.337 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg3 2023-09-29T18:28:59.363 INFO:tasks.workunit.client.0.smithi143.stdout:252019b3cb59 testimg3 2023-09-29T18:28:59.364 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg5 2023-09-29T18:28:59.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:28:59.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:28:59.427 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:28:59.453 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:28:59.454 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:28:59.454 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:00.989 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:00.989 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:00.989 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:00.990 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:29:01.016 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:29:01.016 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:01.016 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg3 2023-09-29T18:29:01.043 INFO:tasks.workunit.client.0.smithi143.stdout:252019b3cb59 testimg3 2023-09-29T18:29:01.043 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:29:01.043 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{ print $1 }' 2023-09-29T18:29:01.071 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=252019b3cb59 2023-09-29T18:29:01.071 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 252019b3cb59 2023-09-29T18:29:01.833 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:29:01.843 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:01.906 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:29:01.909 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:01.909 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:01.909 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:29:01.936 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:01.936 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:29:01.983 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1@snap 2023-09-29T18:29:02.620 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:02.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-09-29T18:29:02.706 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg1@snap 2023-09-29T18:29:02.749 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:02.758 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:29:02.802 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap 2023-09-29T18:29:03.844 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:03.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-09-29T18:29:03.934 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-09-29T18:29:04.007 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg2@snap 2023-09-29T18:29:04.056 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:04.067 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg4@snap 2023-09-29T18:29:04.846 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:04.855 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-09-29T18:29:04.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg4@snap 2023-09-29T18:29:04.994 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:05.004 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-09-29T18:29:05.086 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:05.088 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-09-29T18:29:05.165 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:05.168 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:29:05.233 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-09-29T18:29:05.313 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:05.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:29:05.343 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:29:05.343 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:05.343 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:05.562 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:05.563 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.563 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:05.563 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:29:05.590 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:29:05.590 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.590 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:29:05.617 INFO:tasks.workunit.client.0.smithi143.stdout:2580ac594769 testimg1 2023-09-29T18:29:05.617 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.617 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:29:05.644 INFO:tasks.workunit.client.0.smithi143.stdout:25869bd2f9ef testimg2 2023-09-29T18:29:05.644 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.644 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg4 2023-09-29T18:29:05.671 INFO:tasks.workunit.client.0.smithi143.stdout:2591a5cd13f8 testimg4 2023-09-29T18:29:05.671 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg6 2023-09-29T18:29:05.732 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:05.732 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:05.732 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:29:05.759 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:29:05.759 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:05.760 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:06.955 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:06.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:06.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:06.956 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 2 2023-09-29T18:29:06.983 INFO:tasks.workunit.client.0.smithi143.stdout:2 2023-09-29T18:29:06.983 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:06.983 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:29:07.010 INFO:tasks.workunit.client.0.smithi143.stdout:2580ac594769 testimg1 2023-09-29T18:29:07.010 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:07.010 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:29:07.037 INFO:tasks.workunit.client.0.smithi143.stdout:25869bd2f9ef testimg2 2023-09-29T18:29:07.037 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg5 2023-09-29T18:29:07.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:07.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:07.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:29:07.131 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:29:07.133 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:07.227 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:07.227+0000 7f23c27fc700 0 -- 172.21.15.143:0/2816706731 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5622802477a0 msgr2=0x5622802f08f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:07.237 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:07.236+0000 7f23c27fc700 0 -- 172.21.15.143:0/2816706731 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f23a403a7b0 msgr2=0x7f23a403cbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:08.696 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:29:08.699 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:08.699 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:08.699 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:29:08.727 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:08.727 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-09-29T18:29:08.774 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1@snap 2023-09-29T18:29:09.620 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:09.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-09-29T18:29:09.702 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg1@snap 2023-09-29T18:29:09.745 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:09.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-09-29T18:29:09.800 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg3@snap 2023-09-29T18:29:10.625 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:10.635 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap 2023-09-29T18:29:11.630 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:11.642 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-09-29T18:29:11.731 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-09-29T18:29:11.808 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg2@snap 2023-09-29T18:29:11.857 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:11.867 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg4@snap 2023-09-29T18:29:12.624 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:12.634 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-09-29T18:29:12.724 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm testimg4@snap 2023-09-29T18:29:12.785 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:12.794 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-09-29T18:29:12.877 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:12.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-09-29T18:29:12.961 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:12.964 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg3 2023-09-29T18:29:13.024 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-09-29T18:29:13.101 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:13.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:13.104 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:29:13.130 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:29:13.131 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:13.131 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:13.268 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:13.269 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.270 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:13.271 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:29:13.296 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:29:13.296 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg6 2023-09-29T18:29:13.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.363 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:13.364 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 5 2023-09-29T18:29:13.390 INFO:tasks.workunit.client.0.smithi143.stdout:5 2023-09-29T18:29:13.391 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:13.391 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:13.765 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:13.766 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.766 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:13.766 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 3 2023-09-29T18:29:13.792 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:29:13.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.793 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg1 2023-09-29T18:29:13.819 INFO:tasks.workunit.client.0.smithi143.stdout:25df71295bd8 testimg1 2023-09-29T18:29:13.820 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.820 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg2 2023-09-29T18:29:13.846 INFO:tasks.workunit.client.0.smithi143.stdout:25e5c507a0e testimg2 2023-09-29T18:29:13.847 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.847 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg3 2023-09-29T18:29:13.873 INFO:tasks.workunit.client.0.smithi143.stdout:25eb33622fb7 testimg3 2023-09-29T18:29:13.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash mv testimg5 2023-09-29T18:29:13.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:13.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:13.942 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 4 2023-09-29T18:29:13.968 INFO:tasks.workunit.client.0.smithi143.stdout:4 2023-09-29T18:29:13.968 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:13.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'some expired images could not be removed' 2023-09-29T18:29:15.740 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: some expired images could not be removed 2023-09-29T18:29:15.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:15.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:15.741 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:29:15.767 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:29:15.768 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:15.768 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep testimg3 2023-09-29T18:29:15.794 INFO:tasks.workunit.client.0.smithi143.stdout:25eb33622fb7 testimg3 2023-09-29T18:29:15.795 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd trash ls 2023-09-29T18:29:15.795 INFO:tasks.workunit.client.0.smithi143.stderr:++ awk '{ print $1 }' 2023-09-29T18:29:15.822 INFO:tasks.workunit.client.0.smithi143.stderr:+ ID=25eb33622fb7 2023-09-29T18:29:15.822 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge --image-id 25eb33622fb7 2023-09-29T18:29:16.633 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:29:16.649 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash purge 2023-09-29T18:29:16.714 INFO:tasks.workunit.client.0.smithi143.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-09-29T18:29:16.716 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls 2023-09-29T18:29:16.716 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:16.717 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 0 2023-09-29T18:29:16.743 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:16.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_deep_copy_clone 2023-09-29T18:29:16.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing deep copy clone...' 2023-09-29T18:29:16.744 INFO:tasks.workunit.client.0.smithi143.stdout:testing deep copy clone... 2023-09-29T18:29:16.745 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:29:16.745 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:16.827 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:16.910 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:16.992 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.075 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.358 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.438 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.522 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.606 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.698 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:17.984 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.067 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.150 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.231 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.316 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.398 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.484 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.569 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:18.653 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create testimg1 --image-format 2 --size 256 2023-09-29T18:29:18.698 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg1 --snap=snap1 2023-09-29T18:29:19.650 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:19.663 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect testimg1@snap1 2023-09-29T18:29:19.705 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone testimg1@snap1 testimg2 2023-09-29T18:29:19.774 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap2 2023-09-29T18:29:20.649 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:20.660 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy testimg2 testimg3 2023-09-29T18:29:20.759 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:20.759+0000 7fb0b41ba700 0 -- 172.21.15.143:0/622458127 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55a41d3ed8a0 msgr2=0x55a41d4dae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:21.666 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete...2023-09-29T18:29:21.667+0000 7fb0b41ba700 0 -- 172.21.15.143:0/622458127 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7fb09803a680 msgr2=0x7fb09803caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:21.667 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 100% complete...done. 2023-09-29T18:29:21.671 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:29:21.671 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:29:21.710 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:29:21.711 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:29:21.711 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'parent: rbd/testimg1@snap1' 2023-09-29T18:29:21.752 INFO:tasks.workunit.client.0.smithi143.stdout: parent: rbd/testimg1@snap1 2023-09-29T18:29:21.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg3 2023-09-29T18:29:21.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:29:21.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:21.753 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:29:21.794 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:29:21.795 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg3 2023-09-29T18:29:21.795 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '.*snap2.*' 2023-09-29T18:29:21.834 INFO:tasks.workunit.client.0.smithi143.stdout: 76 snap2 256 MiB Fri Sep 29 18:29:21 2023 2023-09-29T18:29:21.835 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg2 2023-09-29T18:29:21.835 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'features:.*deep-flatten' 2023-09-29T18:29:21.875 INFO:tasks.workunit.client.0.smithi143.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-09-29T18:29:21.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:29:21.876 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'features:.*deep-flatten' 2023-09-29T18:29:21.915 INFO:tasks.workunit.client.0.smithi143.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-09-29T18:29:21.915 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd flatten testimg2 2023-09-29T18:29:21.967 INFO:tasks.workunit.client.0.smithi143.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-09-29T18:29:21.977 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd flatten testimg3 2023-09-29T18:29:22.027 INFO:tasks.workunit.client.0.smithi143.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-09-29T18:29:22.035 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect testimg1@snap1 2023-09-29T18:29:22.080 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge testimg2 2023-09-29T18:29:22.653 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:29:22.664 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap purge testimg3 2023-09-29T18:29:23.653 INFO:tasks.workunit.client.0.smithi143.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-09-29T18:29:23.664 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg2 2023-09-29T18:29:23.791 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-09-29T18:29:23.791+0000 7fc2a6a44700 0 -- 172.21.15.143:0/1459173498 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fc28c008740 msgr2=0x7fc28c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:23.791 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:23.794 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm testimg3 2023-09-29T18:29:23.905 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-09-29T18:29:23.906+0000 7f8ccc2d6700 0 -- 172.21.15.143:0/44840454 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x557588b4e780 msgr2=0x557588b4ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:23.906 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:23.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect testimg1@snap1 2023-09-29T18:29:23.950 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone testimg1@snap1 testimg2 2023-09-29T18:29:24.017 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create testimg2@snap2 2023-09-29T18:29:24.669 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:24.680 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd deep copy --flatten testimg2 testimg3 2023-09-29T18:29:25.770 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 1% complete...2023-09-29T18:29:25.771+0000 7fc9c7361700 0 -- 172.21.15.143:0/3281514881 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fc9a00031a0 msgr2=0x7fc9a0011cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:25.868 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:25.869+0000 7fc9c7b62700 0 -- 172.21.15.143:0/3281514881 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55b9512fabe0 msgr2=0x55b95135be60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:29:26.466 INFO:tasks.workunit.client.0.smithi143.stderr: Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-09-29T18:29:26.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:29:26.469 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'size 256 MiB' 2023-09-29T18:29:26.503 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:29:26.504 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg3 2023-09-29T18:29:26.504 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v parent: 2023-09-29T18:29:26.538 INFO:tasks.workunit.client.0.smithi143.stdout:rbd image 'testimg3': 2023-09-29T18:29:26.538 INFO:tasks.workunit.client.0.smithi143.stdout: size 256 MiB in 64 objects 2023-09-29T18:29:26.538 INFO:tasks.workunit.client.0.smithi143.stdout: order 22 (4 MiB objects) 2023-09-29T18:29:26.538 INFO:tasks.workunit.client.0.smithi143.stdout: snapshot_count: 1 2023-09-29T18:29:26.538 INFO:tasks.workunit.client.0.smithi143.stdout: id: 26f9d4f92729 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: block_name_prefix: rbd_data.26f9d4f92729 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: format: 2 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: op_features: 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: flags: 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: create_timestamp: Fri Sep 29 18:29:24 2023 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: access_timestamp: Fri Sep 29 18:29:24 2023 2023-09-29T18:29:26.539 INFO:tasks.workunit.client.0.smithi143.stdout: modify_timestamp: Fri Sep 29 18:29:24 2023 2023-09-29T18:29:26.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg3 2023-09-29T18:29:26.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -v SNAPID 2023-09-29T18:29:26.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:26.540 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 1 2023-09-29T18:29:26.574 INFO:tasks.workunit.client.0.smithi143.stdout:1 2023-09-29T18:29:26.575 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap ls testimg3 2023-09-29T18:29:26.575 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '.*snap2.*' 2023-09-29T18:29:26.610 INFO:tasks.workunit.client.0.smithi143.stdout: 80 snap2 256 MiB Fri Sep 29 18:29:25 2023 2023-09-29T18:29:26.610 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd info testimg2 2023-09-29T18:29:26.610 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'features:.*deep-flatten' 2023-09-29T18:29:26.657 INFO:tasks.workunit.client.0.smithi143.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-09-29T18:29:26.657 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd flatten testimg2 2023-09-29T18:29:26.712 INFO:tasks.workunit.client.0.smithi143.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-09-29T18:29:26.716 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect testimg1@snap1 2023-09-29T18:29:26.761 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:29:26.761 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:27.755 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:28.847 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:29.913 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:29.997 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.080 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.164 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.247 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.333 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.418 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.502 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.593 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.677 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.767 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.853 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:30.935 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.021 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.107 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.190 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_clone_v2 2023-09-29T18:29:31.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing clone v2...' 2023-09-29T18:29:31.191 INFO:tasks.workunit.client.0.smithi143.stdout:testing clone v2... 2023-09-29T18:29:31.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:29:31.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.274 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.356 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.444 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.528 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.610 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.693 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.777 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:31.863 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.146 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.429 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.511 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.595 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.683 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.765 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.848 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:32.930 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:33.013 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:33.095 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:29:33.142 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@1 2023-09-29T18:29:33.684 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:33.695 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test2 2023-09-29T18:29:33.731 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:33.731+0000 7f1a3af37700 -1 librbd::image::CloneRequest: 0x560ca4cd37b0 validate_parent: parent snapshot must be protected 2023-09-29T18:29:33.731 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: clone error: (22) Invalid argument 2023-09-29T18:29:33.734 INFO:tasks.workunit.client.0.smithi143.stderr:+ true 2023-09-29T18:29:33.734 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test2 2023-09-29T18:29:33.808 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test3 2023-09-29T18:29:33.880 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap protect test1@1 2023-09-29T18:29:33.923 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test4 2023-09-29T18:29:34.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd children test1@1 2023-09-29T18:29:34.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ sort 2023-09-29T18:29:34.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr '\n' ' ' 2023-09-29T18:29:34.001 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -E 'test2.*test3.*test4' 2023-09-29T18:29:34.042 INFO:tasks.workunit.client.0.smithi143.stdout:rbd/test2 rbd/test3 rbd/test4 2023-09-29T18:29:34.043 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd children --descendants test1 2023-09-29T18:29:34.043 INFO:tasks.workunit.client.0.smithi143.stderr:+ sort 2023-09-29T18:29:34.043 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr '\n' ' ' 2023-09-29T18:29:34.043 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -E 'test2.*test3.*test4' 2023-09-29T18:29:34.094 INFO:tasks.workunit.client.0.smithi143.stdout:rbd/test2 rbd/test3 rbd/test4 2023-09-29T18:29:34.095 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd remove test4 2023-09-29T18:29:34.213 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:34.214+0000 7f472f804700 0 -- 172.21.15.143:0/2417411303 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55d6711bd7b0 msgr2=0x55d671263c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:34.216 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:34.219 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap unprotect test1@1 2023-09-29T18:29:34.265 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap remove test1@1 2023-09-29T18:29:34.309 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:34.320 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap list --all test1 2023-09-29T18:29:34.320 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep -E 'trash \(1\) *$' 2023-09-29T18:29:34.355 INFO:tasks.workunit.client.0.smithi143.stdout: 84 5253bb7a-eaed-45bf-8ecd-4e9006e85ad4 1 MiB Fri Sep 29 18:29:33 2023 trash (1) 2023-09-29T18:29:34.355 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@2 2023-09-29T18:29:34.684 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:34.694 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:34.695 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'image has snapshots' 2023-09-29T18:29:34.754 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image has snapshots - these must be deleted with 'rbd snap purge' before the image can be removed. 2023-09-29T18:29:34.754 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm test1@2 2023-09-29T18:29:35.681 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:35.702 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:35.703 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'linked clones' 2023-09-29T18:29:35.763 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-09-29T18:29:35.764 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test3 2023-09-29T18:29:35.875 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:35.875+0000 7fef6ff29700 0 -- 172.21.15.143:0/741644175 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fef4c008740 msgr2=0x7fef4c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:35.888 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:35.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:35.891 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep 'linked clones' 2023-09-29T18:29:35.951 INFO:tasks.workunit.client.0.smithi143.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-09-29T18:29:35.951 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd flatten test2 2023-09-29T18:29:36.700 INFO:tasks.workunit.client.0.smithi143.stderr: Image flatten: 100% complete...done. 2023-09-29T18:29:36.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap list --all test1 2023-09-29T18:29:36.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:36.712 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:29:36.748 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:36.749 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:36.870 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:36.871+0000 7facbdf9f700 0 -- 172.21.15.143:0/3626527292 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55a8ce70e350 msgr2=0x55a8ce7ae6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:36.875 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:36.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:29:37.004 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:37.005+0000 7f7efcd73700 0 -- 172.21.15.143:0/3115874526 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f7ed8006530 msgr2=0x7f7ed80148c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:37.005 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:37.008 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 -s 1 test1 2023-09-29T18:29:37.053 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@1 2023-09-29T18:29:37.629 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:37.639 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create test1@2 2023-09-29T18:29:38.698 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:29:38.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test1@1 test2 --rbd-default-clone-format 2 2023-09-29T18:29:38.787 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone test1@2 test3 --rbd-default-clone-format 2 2023-09-29T18:29:38.857 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm test1@1 2023-09-29T18:29:38.904 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:38.915 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm test1@2 2023-09-29T18:29:38.959 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:29:38.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd rm test1 2023-09-29T18:29:38.969 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:39.023 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:39.023+0000 7fa704452500 -1 librbd::api::Image: remove: image has snapshots - not removing 2023-09-29T18:29:39.023 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 0% complete...failed. 2023-09-29T18:29:39.029 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-09-29T18:29:39.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:29:39.032 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 --rbd-move-parent-to-trash-on-remove=true 2023-09-29T18:29:39.118 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:39.121 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -a 2023-09-29T18:29:39.121 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:39.149 INFO:tasks.workunit.client.0.smithi143.stdout:282e77dafb69 test1 2023-09-29T18:29:39.149 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test2 2023-09-29T18:29:39.258 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:39.259+0000 7fa719c84700 0 -- 172.21.15.143:0/3288724844 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fa6f4001eb0 msgr2=0x7fa6f4008020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:39.685 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:39.685+0000 7fa719c84700 0 -- 172.21.15.143:0/3288724844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555d8a68f2c0 msgr2=0x555d8a71c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:39.728 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:39.731 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -a 2023-09-29T18:29:39.731 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:39.759 INFO:tasks.workunit.client.0.smithi143.stdout:282e77dafb69 test1 2023-09-29T18:29:39.759 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test3 2023-09-29T18:29:39.872 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:39.872+0000 7f8f601cf700 0 -- 172.21.15.143:0/3855242315 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x560ed58ecbc0 msgr2=0x560ed58f1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:40.686 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:40.687+0000 7f8f601cf700 0 -- 172.21.15.143:0/3855242315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ed57b82c0 msgr2=0x560ed5845780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:40.766 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:40.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd trash ls -a 2023-09-29T18:29:40.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail grep test1 2023-09-29T18:29:40.769 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:40.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:29:40.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_thick_provision 2023-09-29T18:29:40.798 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing thick provision...' 2023-09-29T18:29:40.798 INFO:tasks.workunit.client.0.smithi143.stdout:testing thick provision... 2023-09-29T18:29:40.799 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:29:40.799 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:40.882 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:40.967 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.054 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.137 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.221 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.304 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.388 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.471 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.556 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.640 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.729 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.814 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.898 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:41.981 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:42.064 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:42.147 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:42.232 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:42.315 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --thick-provision -s 64M test1 2023-09-29T18:29:42.617 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-09-29T18:29:42.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ count=0 2023-09-29T18:29:42.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret= 2023-09-29T18:29:42.629 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 -lt 10 ']' 2023-09-29T18:29:42.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:29:42.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:42.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr -s ' ' 2023-09-29T18:29:42.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ cut -d ' ' -f 4-5 2023-09-29T18:29:42.630 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^64 MiB' 2023-09-29T18:29:42.666 INFO:tasks.workunit.client.0.smithi143.stdout:64 MiB 2023-09-29T18:29:42.666 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret=0 2023-09-29T18:29:42.666 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 = 0 ']' 2023-09-29T18:29:42.666 INFO:tasks.workunit.client.0.smithi143.stderr:+ break 2023-09-29T18:29:42.667 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:29:42.697 INFO:tasks.workunit.client.0.smithi143.stdout:NAME PROVISIONED USED 2023-09-29T18:29:42.697 INFO:tasks.workunit.client.0.smithi143.stdout:test1 64 MiB 64 MiB 2023-09-29T18:29:42.699 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 '!=' 0 ']' 2023-09-29T18:29:42.700 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:42.814 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2023-09-29T18:29:42.814+0000 7f4136feb700 0 -- 172.21.15.143:0/2020397531 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f4114008740 msgr2=0x7f411400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:42.846 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:29:42.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:29:42.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:42.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:42.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:29:42.878 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:42.878 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --thick-provision -s 4G test1 2023-09-29T18:29:43.383 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete...2023-09-29T18:29:43.383+0000 7f4f7847d700 0 -- 172.21.15.143:0/4187435344 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x55e5e459ed70 msgr2=0x55e5e458fad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:43.526 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 6% complete...2023-09-29T18:29:43.526+0000 7f4f77c7c700 0 -- 172.21.15.143:0/4187435344 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x55e5e468f7e0 msgr2=0x55e5e4691c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:53.665 INFO:tasks.workunit.client.0.smithi143.stderr: Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete... Thick provisioning: 12% complete... Thick provisioning: 13% complete... Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-09-29T18:29:53.677 INFO:tasks.workunit.client.0.smithi143.stderr:+ count=0 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret= 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 -lt 10 ']' 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ tr -s ' ' 2023-09-29T18:29:53.678 INFO:tasks.workunit.client.0.smithi143.stderr:+ cut -d ' ' -f 4-5 2023-09-29T18:29:53.679 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^4 GiB' 2023-09-29T18:29:53.712 INFO:tasks.workunit.client.0.smithi143.stdout:4 GiB 2023-09-29T18:29:53.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ ret=0 2023-09-29T18:29:53.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 = 0 ']' 2023-09-29T18:29:53.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ break 2023-09-29T18:29:53.713 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd du 2023-09-29T18:29:53.744 INFO:tasks.workunit.client.0.smithi143.stdout:NAME PROVISIONED USED 2023-09-29T18:29:53.744 INFO:tasks.workunit.client.0.smithi143.stdout:test1 4 GiB 4 GiB 2023-09-29T18:29:53.746 INFO:tasks.workunit.client.0.smithi143.stderr:+ '[' 0 '!=' 0 ']' 2023-09-29T18:29:53.746 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm test1 2023-09-29T18:29:53.878 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 2% complete...2023-09-29T18:29:53.879+0000 7f8465762700 0 -- 172.21.15.143:0/4061251867 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f84440064e0 msgr2=0x7f84440088c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:53.898 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 3% complete... Removing image: 4% complete...2023-09-29T18:29:53.898+0000 7f8465f63700 0 -- 172.21.15.143:0/4061251867 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x559ffa385980 msgr2=0x559ffa42bce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:29:54.904 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-09-29T18:29:54.907 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd ls 2023-09-29T18:29:54.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test1 2023-09-29T18:29:54.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:54.908 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:29:54.937 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:54.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ test_namespace 2023-09-29T18:29:54.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'testing namespace...' 2023-09-29T18:29:54.938 INFO:tasks.workunit.client.0.smithi143.stdout:testing namespace... 2023-09-29T18:29:54.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ remove_images 2023-09-29T18:29:54.938 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.105 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.191 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.277 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.362 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.447 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.533 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.621 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.796 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.881 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:55.966 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.051 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.138 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.223 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.309 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.394 INFO:tasks.workunit.client.0.smithi143.stderr:+ for img in $IMGS 2023-09-29T18:29:56.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace ls 2023-09-29T18:29:56.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:56.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^0$' 2023-09-29T18:29:56.709 INFO:tasks.workunit.client.0.smithi143.stdout:0 2023-09-29T18:29:56.710 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create rbd/test1 2023-09-29T18:29:56.744 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create --pool rbd --namespace test2 2023-09-29T18:29:56.785 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create --namespace test3 2023-09-29T18:29:56.819 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd namespace create rbd/test3 2023-09-29T18:29:56.820 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace create rbd/test3 2023-09-29T18:29:56.849 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: failed to created namespace: (17) File exists 2023-09-29T18:29:56.849 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:56.850+0000 7f76c4b02500 -1 librbd::api::Namespace: create: failed to add namespace: (17) File exists 2023-09-29T18:29:56.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:29:56.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace list 2023-09-29T18:29:56.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep test 2023-09-29T18:29:56.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ wc -l 2023-09-29T18:29:56.852 INFO:tasks.workunit.client.0.smithi143.stderr:+ grep '^3$' 2023-09-29T18:29:56.879 INFO:tasks.workunit.client.0.smithi143.stdout:3 2023-09-29T18:29:56.879 INFO:tasks.workunit.client.0.smithi143.stderr:+ expect_fail rbd namespace remove --pool rbd missing 2023-09-29T18:29:56.879 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd namespace remove --pool rbd missing 2023-09-29T18:29:56.902 INFO:tasks.workunit.client.0.smithi143.stderr:rbd: namespace name was not specified 2023-09-29T18:29:56.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ return 0 2023-09-29T18:29:56.903 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 1G rbd/test1/image1 2023-09-29T18:29:56.957 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-pattern rand --io-total 32M --io-size 4K rbd/test1/image1 2023-09-29T18:29:57.013 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 4096 io_threads 16 bytes 33554432 pattern random 2023-09-29T18:29:58.105 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:29:58.106 INFO:tasks.workunit.client.0.smithi143.stdout: 1 3648 3358.42 13 MiB/s 2023-09-29T18:29:58.110 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:58.111+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f00ec03a690 msgr2=0x7f00ec03cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:29:59.340 INFO:tasks.workunit.client.0.smithi143.stdout: 2 3664 1582.81 6.2 MiB/s 2023-09-29T18:29:59.846 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:59.847+0000 7f0107633700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5612b7f03b10 msgr2=0x7f00ec03d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:29:59.852 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:59.852+0000 7f0107633700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5612b7ea18d0 msgr2=0x7f00ec048230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:29:59.862 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:29:59.861+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f00ec03a690 msgr2=0x7f00ec03cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:00.010 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:00.010+0000 7f0107633700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5612b7f03b10 msgr2=0x7f00ec075dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:30:00.093 INFO:tasks.workunit.client.0.smithi143.stdout: 3 3696 1205.99 4.7 MiB/s 2023-09-29T18:30:00.893 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:00.893+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5612b7ea18d0 msgr2=0x7f00ec03d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:00.907 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:00.907+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5612b7f03b10 msgr2=0x7f00ec056760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:30:01.030 INFO:tasks.workunit.client.0.smithi143.stdout: 4 4992 1247.33 4.9 MiB/s 2023-09-29T18:30:01.303 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:01.303+0000 7f0107633700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f00ec03a690 msgr2=0x7f00ec03ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:01.772 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:01.772+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7f00ec03a690 msgr2=0x7f00ec03ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:02.042 INFO:tasks.workunit.client.0.smithi143.stdout: 5 6544 1304.71 5.1 MiB/s 2023-09-29T18:30:02.138 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:02.138+0000 7f0106e32700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5612b7ea18d0 msgr2=0x7f00ec03d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:03.688 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:03.688+0000 7f0107633700 0 -- 172.21.15.143:0/2478887670 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7f00ec03a690 msgr2=0x7f00ec081b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:03.859 INFO:tasks.workunit.client.0.smithi143.stdout:elapsed: 6 ops: 8192 ops/sec: 1196.62 bytes/sec: 4.7 MiB/s 2023-09-29T18:30:03.874 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap create rbd/test1/image1@1 2023-09-29T18:30:04.805 INFO:tasks.workunit.client.0.smithi143.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-09-29T18:30:04.818 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd clone --rbd-default-clone-format 2 rbd/test1/image1@1 rbd/test2/image1 2023-09-29T18:30:04.937 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd snap rm rbd/test1/image1@1 2023-09-29T18:30:04.985 INFO:tasks.workunit.client.0.smithi143.stderr: Removing snap: 100% complete...done. 2023-09-29T18:30:04.998 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export rbd/test1/image1 - 2023-09-29T18:30:04.998 INFO:tasks.workunit.client.0.smithi143.stderr:+ cmp /dev/fd/63 /dev/fd/62 2023-09-29T18:30:04.998 INFO:tasks.workunit.client.0.smithi143.stderr:++ rbd export rbd/test2/image1 - 2023-09-29T18:30:05.198 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete...2023-09-29T18:30:05.199+0000 7fcfee899700 0 -- 172.21.15.143:0/3883337652 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5607ae440f30 msgr2=0x5607ae443310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:05.238 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete...2023-09-29T18:30:05.239+0000 7fcfee899700 0 -- 172.21.15.143:0/3883337652 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fcfd403a5d0 msgr2=0x7fcfd403c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:05.422 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting imageExporting image: : 3232% complete...% complete... Exporting image: 33 % complete...Exporting image: 33% complete... Exporting image: 34% complete... Exporting imageExporting image: : 3534% complete...% complete... Exporting image: 35% complete...2023-09-29T18:30:05.422+0000 7fafcba42700 0 -- 172.21.15.143:0/1886629167 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5630c277bed0 msgr2=0x5630c2791a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:30:05.476 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete...2023-09-29T18:30:05.477+0000 7fafcb241700 0 -- 172.21.15.143:0/1886629167 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x5630c27df1b0 msgr2=0x5630c27e1590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:06.303 INFO:tasks.workunit.client.0.smithi143.stderr: Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2023-09-29T18:30:06.303 INFO:tasks.workunit.client.0.smithi143.stderr:Exporting image: 100% complete...done. 2023-09-29T18:30:06.310 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd rm rbd/test2/image1 2023-09-29T18:30:06.448 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete...2023-09-29T18:30:06.449+0000 7fc3a50f5700 0 -- 172.21.15.143:0/3641437254 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fc380008740 msgr2=0x7fc38000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T18:30:06.464 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:06.465+0000 7fc3a50f5700 0 -- 172.21.15.143:0/3641437254 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7fc38803a5d0 msgr2=0x7fc38803c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:06.847 INFO:tasks.workunit.client.0.smithi143.stderr: Removing image: 100% complete...done. 2023-09-29T18:30:06.850 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd create --image-format 2 --size 1G rbd/image2 2023-09-29T18:30:07.078 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:07.079+0000 7f855173c700 0 --2- 172.21.15.143:0/2840435890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c385e4ddb0 0x55c385eecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-29T18:30:07.099 INFO:tasks.workunit.client.0.smithi143.stderr:+ rbd bench --io-type write --io-pattern rand --io-total 32M --io-size 4K rbd/image2 2023-09-29T18:30:07.181 INFO:tasks.workunit.client.0.smithi143.stdout:bench type write io_size 4096 io_threads 16 bytes 33554432 pattern random 2023-09-29T18:30:08.214 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T18:30:08.215+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffb6c03a790 msgr2=0x7ffb6c03cbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T18:30:09.997 INFO:tasks.workunit.client.0.smithi143.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-09-29T18:30:09.997 INFO:tasks.workunit.client.0.smithi143.stdout: 2 3200 1142.87 4.5 MiB/s 2023-09-29T18:30:10.436 INFO:tasks.workunit.client.0.smithi143.stdout: 3 3216 993.553 3.9 MiB/s 2023-09-29T18:30:38.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:38.484+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.10617.0:1470 3.3 3:dd3d2e2d:::rbd_data.29767bf0fbb3.000000000000008e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3616768~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:39.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:39.513+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.10617.0:1470 3.3 3:dd3d2e2d:::rbd_data.29767bf0fbb3.000000000000008e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3616768~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:39.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:39.756+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.10617.0:1182 3.2 3:43e4f353:::rbd_data.29767bf0fbb3.000000000000003d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 339968~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:40.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:40.543+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:40.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:40.715+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:41.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:41.538+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:41.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:41.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:42.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:42.508+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:42.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:42.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:43.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:43.535+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:43.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:43.752+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:44.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:44.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:44.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:44.777+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:45.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:45.572+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:45.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:45.732+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:46.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:46.545+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:46.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:46.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:47.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:47.499+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:47.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:47.715+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:48.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:48.545+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:48.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:48.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:49.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:49.535+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:49.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:49.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:50.491 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:50.491+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:50.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:50.744+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:51.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:51.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:51.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:51.769+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:52.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:52.517+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:52.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:52.744+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:53.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:53.563+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:53.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:53.788+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:54.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:54.565+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:54.768 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:54.767+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:55.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:55.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:55.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:55.744+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:56.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:56.569+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:56.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:56.774+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:57.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:57.552+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:57.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:57.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:58.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:58.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:58.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:58.798+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:59.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:30:59.545+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:30:59.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:30:59.771+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:00.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:00.590+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:00.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:00.799+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:01.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:01.637+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:01.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:01.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:02.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:02.615+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:02.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:02.814+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:03.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:03.614+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:03.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:03.859+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:04.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:04.605+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:04.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:04.891+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:05.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:05.571+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:05.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:05.906+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:06.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:06.583+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:06.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:06.896+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:07.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:07.601+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:07.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:07.907+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:08.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:08.598+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:08.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:08.907+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:09.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:09.623+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:09.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:09.904+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:10.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:10.625+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:10.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:10.913+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:11.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:11.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:11.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:11.868+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:12.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:12.711+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:12.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:12.913+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:13.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:13.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:13.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:13.897+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:14.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:14.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:14.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:14.897+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:15.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:15.680+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:15.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:15.934+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:16.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:16.684+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:16.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:16.959+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:18.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:17.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:18.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:17.941+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:18.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:18.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:18.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:18.943+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:19.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:19.724+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:19.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:19.983+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:20.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:20.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:21.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:21.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:21.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:21.639+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:21.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:21.989+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:22.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:22.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:22.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:22.949+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:23.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:23.672+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:23.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:23.934+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:24.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:24.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:24.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:24.949+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:25.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:25.675+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:25.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:25.983+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:26.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:26.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:27.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:27.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:27.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:27.646+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:28.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:28.057+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:28.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:28.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:29.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:29.063+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:29.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:29.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:30.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:30.045+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:30.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:30.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:31.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:31.083+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:31.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:31.653+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:32.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:32.092+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:32.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:32.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:33.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:33.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:33.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:33.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:34.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:34.060+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:34.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:34.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:35.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:35.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:35.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:35.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:36.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:36.057+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:36.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:36.737+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:37.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:37.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:37.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:37.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:38.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:38.078+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:38.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:38.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:39.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:39.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:39.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:39.659+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:40.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:40.123+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:40.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:40.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:41.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:41.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:41.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:41.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:42.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:42.163+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:42.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:42.736+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:43.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:43.154+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:43.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:43.783+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:44.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:44.116+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:44.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:44.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:45.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:45.093+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:45.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:45.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:46.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:46.061+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:46.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:46.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:47.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:47.035+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:47.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:47.844+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:48.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:48.035+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:48.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:48.807+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:49.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:49.022+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:49.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:49.828+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:50.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:50.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:50.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:50.782+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:51.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:51.028+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:51.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:51.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:51.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:51.988+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:52.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:52.750+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:52.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:52.993+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:53.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:53.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:53.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:53.989+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:54.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:54.684+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:54.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:54.953+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:55.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:55.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:55.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:55.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:56.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:56.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:56.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:56.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:57.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:57.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:57.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:57.984+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:58.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:58.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:58.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:31:58.959+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:31:59.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:31:59.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:00.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:00.010+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:00.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:00.771+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:00.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:00.976+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:01.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:01.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:02.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:02.014+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:02.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:02.797+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:02.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:02.973+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:03.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:03.822+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:03.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:03.985+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:04.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:04.857+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:04.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:04.991+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:05.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:05.884+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:05.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:05.996+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:06.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:06.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:06.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:06.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:07.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:07.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:07.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:07.998+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:08.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:08.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:09.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:09.048+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:09.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:09.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:10.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:10.007+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:10.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:10.829+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:11.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:11.011+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:11.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:11.826+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:12.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:12.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:12.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:12.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:13.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:13.006+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:13.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:13.859+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:14.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:14.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:14.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:14.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:15.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:15.013+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:15.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:15.820+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:15.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:15.974+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:16.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:16.810+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:17.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:17.020+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:17.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:17.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:18.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:18.017+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:18.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:18.806+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:18.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:18.990+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:19.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:19.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:20.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:20.032+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:20.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:20.844+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:21.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:21.019+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:21.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:21.845+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:21.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:21.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:22.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:22.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:23.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:23.026+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:23.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:23.867+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:24.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:24.015+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:24.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:24.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:25.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:25.040+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:25.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:25.847+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:26.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:26.030+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:26.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:26.863+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:27.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:27.048+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:27.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:27.852+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:28.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:28.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:28.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:28.896+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:29.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:29.054+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:29.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:29.892+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:30.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:30.037+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:30.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:30.858+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:31.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:31.039+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:31.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:31.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:32.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:32.023+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:32.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:32.926+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:33.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:33.064+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:33.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:33.969+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:34.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:34.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:34.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:34.976+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:35.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:35.065+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:35.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:35.938+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:36.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:36.095+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:36.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:36.984+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:37.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:37.080+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:38.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:38.008+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:38.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:38.113+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:39.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:39.054+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:39.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:39.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:40.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:40.048+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:40.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:40.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:41.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:41.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:41.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:41.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:42.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:42.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:42.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:42.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:43.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:43.002+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:43.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:43.104+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:44.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:44.006+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:44.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:44.094+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:45.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:45.008+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:45.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:45.123+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:46.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:46.057+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:46.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:46.117+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:47.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:47.064+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:47.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:47.081+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:48.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:48.055+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:48.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:48.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:49.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:49.023+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:49.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:49.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:50.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:50.065+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:50.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:50.169+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:51.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:51.079+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:51.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:51.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:52.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:52.091+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:52.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:52.160+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:53.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:53.068+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:53.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:53.166+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:54.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:54.059+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:54.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:54.183+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:55.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:55.090+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:55.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:55.164+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:56.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:56.081+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:56.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:56.174+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:57.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:57.045+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:57.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:57.205+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:58.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:58.013+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:58.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:58.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:58.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:32:58.985+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:32:59.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:32:59.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:00.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:00.018+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:00.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:00.271+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:01.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:01.032+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:01.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:01.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:01.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:01.986+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:02.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:02.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:02.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:02.951+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:03.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:03.381+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:03.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:03.999+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:04.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:04.337+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:05.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:05.044+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:05.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:05.359+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:06.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:06.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:06.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:06.401+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:06.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:06.965+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:07.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:07.426+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:07.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:07.999+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:08.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:08.426+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:09.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:09.005+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:09.426 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:09.427+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:09.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:09.971+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:10.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:10.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:10.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:10.933+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:11.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:11.449+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:11.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:11.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:12.490 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:12.491+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:12.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:12.873+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:13.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:13.538+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:13.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:13.847+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:14.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:14.535+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:14.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:14.832+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:15.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:15.558+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:15.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:15.826+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:16.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:16.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:16.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:16.801+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:17.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:17.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:17.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:17.774+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:18.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:18.601+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:18.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:18.794+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:19.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:19.641+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:19.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:19.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:20.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:20.603+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:20.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:20.803+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:21.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:21.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:21.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:21.825+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:22.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:22.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:22.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:22.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:23.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:23.477+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:23.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:23.853+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:24.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:24.431+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:24.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:24.820+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:25.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:25.465+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:25.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:25.785+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:26.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:26.443+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:26.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:26.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:27.470 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:27.470+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:27.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:27.747+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:28.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:28.493+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:28.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:28.741+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:29.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:29.451+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:29.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:29.692+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:30.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:30.481+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:30.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:30.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:31.443 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:31.443+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:31.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:31.675+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:32.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:32.463+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:32.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:32.636+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:33.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:33.437+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:33.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:33.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:34.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:34.431+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:34.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:34.608+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:35.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:35.392+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:35.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:35.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:36.423 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:36.424+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:36.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:36.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:37.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:37.433+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:37.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:37.720+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:38.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:38.389+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:38.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:38.691+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:39.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:39.398+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:39.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:39.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:40.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:40.442+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:40.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:40.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:41.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:41.400+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:41.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:41.757+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:42.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:42.417+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:42.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:42.741+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:43.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:43.420+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:43.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:43.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:44.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:44.384+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:44.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:44.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:45.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:45.354+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:45.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:45.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:46.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:46.342+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:46.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:46.719+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:47.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:47.381+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:47.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:47.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:48.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:48.333+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:48.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:48.763+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:49.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:49.346+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:49.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:49.805+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:50.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:50.343+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:50.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:50.854+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:51.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:51.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:51.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:51.889+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:52.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:52.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:52.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:52.935+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:53.310 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:53.311+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:53.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:53.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:54.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:54.332+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:54.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:54.913+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:55.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:55.345+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:55.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:55.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:56.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:56.330+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:56.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:56.971+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:57.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:57.332+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:58.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:58.016+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:58.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:58.325+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:58.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:58.996+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:59.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:33:59.294+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:33:59.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:33:59.964+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:00.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:00.258+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:00.977 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:00.976+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:01.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:01.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:01.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:01.971+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:02.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:02.269+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:02.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:02.980+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:03.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:03.283+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:03.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:03.974+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:04.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:04.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:04.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:04.964+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:05.335 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:05.335+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:05.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:05.929+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:06.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:06.358+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:06.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:06.967+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:07.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:07.376+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:07.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:07.974+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:08.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:08.382+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:08.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:08.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:09.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:09.396+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:09.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:09.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:10.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:10.402+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:10.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:10.891+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:11.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:11.439+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:11.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:11.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:12.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:12.439+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:12.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:12.815+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:13.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:13.480+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:13.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:13.828+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:14.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:14.502+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:14.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:14.825+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:15.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:15.488+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:15.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:15.824+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:16.451 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:16.451+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:16.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:16.812+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:17.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:17.422+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:17.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:17.858+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:18.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:18.390+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:18.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:18.855+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:19.394 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:19.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:19.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:19.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:20.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:20.440+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:20.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:20.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:21.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:21.480+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:21.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:21.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:22.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:22.436+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:22.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:22.868+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:23.451 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:23.450+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:23.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:23.868+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:24.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:24.446+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:24.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:24.846+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:25.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:25.421+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:25.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:25.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:26.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:26.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:26.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:26.841+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:27.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:27.404+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:27.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:27.891+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:28.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:28.366+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:28.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:28.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:29.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:29.404+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:29.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:29.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:30.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:30.406+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:30.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:30.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:31.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:31.418+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:31.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:31.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:33.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:32.402+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:33.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:32.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:33.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:33.388+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:33.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:33.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:34.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:34.340+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:34.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:34.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:35.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:35.364+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:35.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:35.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:36.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:36.376+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:36.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:36.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:37.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:37.423+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:37.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:37.898+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:38.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:38.391+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:38.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:38.909+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:39.399 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:39.398+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:39.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:39.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:40.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:40.376+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:40.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:40.872+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:41.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:41.408+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:41.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:41.827+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:42.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:42.366+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:42.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:42.870+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:43.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:43.352+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:43.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:43.870+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:44.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:44.322+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:44.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:44.900+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:45.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:45.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:45.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:45.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:46.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:46.290+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:46.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:46.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:47.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:47.314+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:47.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:47.956+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:48.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:48.360+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:48.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:48.932+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:49.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:49.353+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:49.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:49.951+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:50.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:50.380+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:50.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:50.965+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:51.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:51.332+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:51.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:51.983+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:52.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:52.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:53.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:53.020+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:53.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:53.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:54.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:54.012+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:54.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:54.234+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:55.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:55.003+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:55.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:55.240+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:56.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:56.044+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:56.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:56.196+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:57.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:57.083+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:57.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:57.155+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:58.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:58.096+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:58.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:58.128+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:59.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:34:59.078+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:34:59.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:34:59.131+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:00.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:00.045+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:00.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:00.096+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:01.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:01.013+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:01.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:01.126+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:01.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:01.970+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:02.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:02.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:02.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:02.980+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:03.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:03.116+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:04.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:04.018+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:04.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:04.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:04.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:04.998+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:05.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:05.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:05.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:05.974+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:06.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:06.092+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:06.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:06.958+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:07.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:07.058+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:07.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:07.975+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:08.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:08.108+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:08.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:08.983+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:09.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:09.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:09.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:09.956+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:10.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:10.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:10.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:10.932+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:11.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:11.102+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:11.935 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:11.936+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:12.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:12.101+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:12.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:12.924+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:13.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:13.056+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:13.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:13.962+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:14.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:14.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:14.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:14.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:15.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:15.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:15.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:15.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:16.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:16.164+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:16.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:16.918+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:17.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:17.165+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:17.967 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:17.968+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:18.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:18.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:18.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:18.934+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:19.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:19.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:19.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:19.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:20.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:20.150+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:20.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:20.992+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:21.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:21.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:22.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:22.031+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:22.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:22.166+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:22.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:22.996+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:23.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:23.211+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:23.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:23.972+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:24.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:24.222+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:24.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:24.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:25.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:25.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:25.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:25.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:26.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:26.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:27.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:27.032+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:27.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:27.112+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:28.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:28.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:28.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:28.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:29.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:29.085+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:29.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:29.145+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:30.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:30.111+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:30.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:30.112+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:31.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:31.094+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:31.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:31.138+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:32.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:32.132+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:32.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:32.173+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:33.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:33.126+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:33.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:33.152+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:34.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:34.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:34.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:34.165+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:35.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:35.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:35.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:35.156+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:36.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:36.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:36.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:36.179+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:37.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:37.098+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:37.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:37.216+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:38.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:38.097+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:38.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:38.255+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:39.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:39.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:39.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:39.274+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:40.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:40.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:40.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:40.225+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:41.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:41.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:41.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:41.244+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:42.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:42.160+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:42.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:42.288+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:43.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:43.185+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:43.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:43.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:44.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:44.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:44.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:44.258+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:45.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:45.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:45.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:45.273+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:46.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:46.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:46.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:46.226+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:47.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:47.074+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:47.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:47.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:48.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:48.094+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:48.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:48.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:49.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:49.058+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:49.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:49.188+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:50.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:50.093+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:50.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:50.166+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:51.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:51.098+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:51.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:51.172+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:52.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:52.078+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:52.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:52.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:53.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:53.046+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:53.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:53.203+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:54.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:54.010+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:54.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:54.178+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:55.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:55.043+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:55.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:55.157+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:56.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:56.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:56.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:56.148+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:57.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:57.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:57.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:57.131+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:58.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:58.062+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:58.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:58.176+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:59.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:35:59.064+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:35:59.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:35:59.215+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:00.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:00.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:00.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:00.249+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:01.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:01.024+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:01.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:01.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:02.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:02.071+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:02.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:02.250+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:03.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:03.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:03.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:03.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:04.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:04.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:04.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:04.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:05.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:05.124+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:05.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:05.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:06.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:06.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:06.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:06.325+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:07.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:07.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:07.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:07.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:08.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:08.141+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:08.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:08.374+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:09.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:09.179+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:09.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:09.340+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:10.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:10.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:10.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:10.300+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:11.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:11.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:11.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:11.262+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:12.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:12.279+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:12.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:12.307+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:13.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:13.238+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:13.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:13.283+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:14.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:14.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:14.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:14.300+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:15.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:15.183+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:15.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:15.281+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:16.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:16.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:16.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:16.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:17.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:17.195+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:17.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:17.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:18.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:18.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:18.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:18.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:19.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:19.215+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:19.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:19.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:20.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:20.203+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:20.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:20.289+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:21.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:21.163+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:21.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:21.261+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:22.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:22.200+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:22.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:22.243+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:23.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:23.196+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:23.227 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:23.227+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:24.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:24.206+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:24.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:24.207+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:25.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:25.169+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:25.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:25.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:26.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:26.186+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:26.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:26.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:27.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:27.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:27.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:27.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:28.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:28.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:28.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:28.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:29.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:29.187+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:29.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:29.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:30.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:30.214+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:30.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:30.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:31.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:31.263+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:31.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:31.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:32.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:32.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:32.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:32.263+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:33.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:33.238+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:33.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:33.291+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:34.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:34.257+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:34.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:34.263+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:35.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:35.257+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:35.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:35.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:36.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:36.255+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:36.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:36.260+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:37.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:37.253+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:37.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:37.263+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:38.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:38.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:38.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:38.273+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:39.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:39.248+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:39.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:39.305+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:40.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:40.264+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:40.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:40.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:41.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:41.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:41.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:41.289+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:42.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:42.233+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:42.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:42.321+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:43.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:43.237+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:43.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:43.372+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:44.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:44.254+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:44.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:44.386+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:45.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:45.217+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:45.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:45.435+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:46.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:46.175+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:46.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:46.459+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:47.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:47.163+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:47.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:47.471+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:48.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:48.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:48.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:48.509+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:49.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:49.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:49.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:49.541+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:50.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:50.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:50.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:50.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:51.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:51.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:51.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:51.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:52.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:52.196+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:52.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:52.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:53.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:53.191+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:53.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:53.631+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:54.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:54.238+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:54.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:54.617+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:55.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:55.275+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:55.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:55.585+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:56.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:56.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:56.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:56.636+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:57.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:57.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:57.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:57.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:58.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:58.241+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:58.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:58.624+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:59.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:36:59.253+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:36:59.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:36:59.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:00.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:00.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:00.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:00.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:01.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:01.266+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:01.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:01.686+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:02.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:02.250+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:02.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:02.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:03.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:03.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:03.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:03.680+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:04.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:04.280+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:04.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:04.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:05.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:05.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:05.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:05.738+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:06.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:06.352+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:06.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:06.752+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:07.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:07.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:07.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:07.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:08.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:08.346+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:08.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:08.788+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:09.394 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:09.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:09.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:09.810+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:10.435 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:10.436+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:10.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:10.788+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:11.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:11.480+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:11.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:11.802+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:12.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:12.486+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:12.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:12.809+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:13.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:13.521+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:13.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:13.847+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:14.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:14.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:14.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:14.822+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:15.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:15.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:15.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:15.784+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:16.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:16.584+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:16.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:16.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:17.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:17.574+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:17.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:17.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:18.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:18.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:18.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:18.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:19.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:19.591+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:19.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:19.873+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:20.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:20.610+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:20.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:20.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:21.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:21.565+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:21.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:21.896+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:22.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:22.558+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:22.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:22.913+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:23.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:23.590+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:23.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:23.931+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:24.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:24.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:24.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:24.922+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:25.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:25.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:25.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:25.932+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:26.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:26.615+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:26.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:26.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:27.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:27.575+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:27.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:27.860+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:28.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:28.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:28.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:28.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:29.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:29.653+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:29.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:29.896+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:30.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:30.651+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:30.924 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:30.925+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:31.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:31.656+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:31.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:31.892+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:32.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:32.616+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:32.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:32.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:33.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:33.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:33.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:33.912+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:34.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:34.576+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:34.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:34.898+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:35.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:35.594+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:35.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:35.926+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:36.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:36.593+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:36.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:36.922+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:37.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:37.568+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:37.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:37.902+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:38.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:38.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:38.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:38.874+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:39.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:39.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:39.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:39.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:40.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:40.524+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:40.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:40.938+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:41.500 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:41.500+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:41.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:41.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:42.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:42.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:42.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:42.933+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:43.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:43.573+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:43.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:43.915+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:44.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:44.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:44.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:44.872+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:45.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:45.642+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:45.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:45.873+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:46.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:46.636+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:46.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:46.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:47.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:47.654+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:47.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:47.897+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:48.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:48.682+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:48.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:48.858+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:49.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:49.643+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:49.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:49.875+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:50.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:50.664+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:50.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:50.872+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:51.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:51.707+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:51.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:51.831+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:52.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:52.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:52.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:52.837+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:53.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:53.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:53.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:53.885+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:54.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:54.639+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:54.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:54.900+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:55.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:55.681+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:55.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:55.919+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:56.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:56.711+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:56.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:56.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:57.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:57.686+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:57.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:57.985+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:58.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:58.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:58.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:58.938+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:59.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:37:59.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:37:59.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:37:59.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:00.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:00.711+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:00.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:00.975+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:01.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:01.714+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:02.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:02.019+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:02.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:02.746+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:02.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:02.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:03.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:03.764+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:04.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:04.025+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:04.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:04.813+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:05.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:05.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:05.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:05.797+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:06.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:06.071+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:06.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:06.825+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:07.063 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:07.062+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:07.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:07.779+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:08.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:08.016+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:08.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:08.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:09.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:09.001+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:09.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:09.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:09.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:09.985+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:10.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:10.723+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:10.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:10.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:11.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:11.763+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:11.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:11.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:12.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:12.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:12.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:12.894+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:13.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:13.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:13.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:13.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:14.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:14.783+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:14.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:14.824+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:15.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:15.737+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:15.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:15.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:16.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:16.689+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:16.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:16.844+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:17.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:17.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:17.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:17.828+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:18.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:18.663+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:18.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:18.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:19.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:19.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:19.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:19.812+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:20.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:20.571+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:20.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:20.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:21.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:21.570+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:21.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:21.865+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:22.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:22.535+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:22.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:22.908+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:23.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:23.568+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:23.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:23.957+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:24.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:24.557+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:24.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:24.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:25.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:25.513+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:25.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:25.931+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:26.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:26.493+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:26.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:26.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:27.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:27.503+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:27.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:27.881+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:28.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:28.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:28.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:28.925+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:29.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:29.564+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:29.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:29.919+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:30.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:30.551+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:30.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:30.915+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:31.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:31.567+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:31.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:31.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:32.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:32.613+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:32.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:32.955+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:33.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:33.604+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:33.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:33.959+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:34.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:34.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:34.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:34.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:35.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:35.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:35.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:35.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:36.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:36.521+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:36.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:36.918+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:37.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:37.493+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:37.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:37.885+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:38.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:38.516+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:38.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:38.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:39.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:39.536+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:39.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:39.946+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:40.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:40.543+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:40.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:40.976+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:41.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:41.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:42.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:41.999+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:42.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:42.519+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:42.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:42.953+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:43.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:43.496+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:43.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:43.949+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:44.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:44.510+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:44.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:44.986+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:45.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:45.477+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:46.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:46.030+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:46.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:46.520+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:47.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:47.075+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:47.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:47.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:48.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:48.045+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:48.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:48.502+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:49.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:49.020+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:49.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:49.488+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:50.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:50.008+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:50.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:50.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:51.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:51.026+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:51.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:51.493+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:52.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:52.053+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:52.495 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:52.494+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:53.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:53.066+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:53.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:53.492+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:54.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:54.070+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:54.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:54.468+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:55.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:55.089+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:55.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:55.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:56.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:56.101+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:56.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:56.566+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:57.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:57.108+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:57.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:57.538+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:58.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:58.071+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:58.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:58.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:59.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:59.024+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:59.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:38:59.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:38:59.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:38:59.980+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:00.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:00.513+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:01.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:01.016+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:01.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:01.472+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:01.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:01.991+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:02.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:02.477+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:03.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:03.002+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:03.467 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:03.468+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:03.967 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:03.968+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:04.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:04.488+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:04.977 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:04.978+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:05.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:05.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:05.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:05.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:06.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:06.526+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:06.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:06.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:07.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:07.488+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:07.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:07.942+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:08.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:08.480+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:08.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:08.952+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:09.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:09.492+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:09.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:09.910+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:10.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:10.505+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:10.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:10.884+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:11.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:11.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:11.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:11.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:12.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:12.574+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:12.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:12.958+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:13.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:13.565+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:13.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:13.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:14.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:14.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:14.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:14.968+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:15.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:15.546+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:15.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:15.972+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:16.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:16.513+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:16.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:16.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:17.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:17.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:17.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:17.992+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:18.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:18.549+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:18.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:18.974+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:19.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:19.532+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:20.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:20.013+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:20.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:20.534+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:21.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:21.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:21.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:21.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:22.055 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:22.056+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:22.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:22.498+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:23.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:23.095+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:23.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:23.473+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:24.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:24.128+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:24.503 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:24.504+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:25.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:25.107+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:25.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:25.540+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:26.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:26.109+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:26.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:26.541+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:27.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:27.065+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:27.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:27.583+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:28.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:28.032+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:28.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:28.578+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:29.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:29.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:29.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:29.616+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:30.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:30.034+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:30.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:30.664+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:31.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:31.050+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:31.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:31.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:32.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:32.031+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:32.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:32.626+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:33.029 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:33.030+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:33.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:33.659+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:34.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:34.037+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:34.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:34.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:35.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:35.082+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:35.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:35.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:36.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:36.041+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:36.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:36.635+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:37.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:37.002+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:37.599 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:37.600+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:38.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:38.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:38.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:38.618+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:39.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:39.067+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:39.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:39.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:40.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:40.045+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:40.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:40.628+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:41.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:41.061+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:41.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:41.658+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:42.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:42.068+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:42.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:42.696+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:43.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:43.054+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:43.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:43.693+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:44.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:44.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:44.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:44.734+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:45.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:45.071+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:45.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:45.686+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:46.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:46.044+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:46.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:46.698+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:47.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:47.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:47.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:47.739+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:48.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:48.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:48.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:48.732+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:49.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:49.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:49.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:49.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:50.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:50.039+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:50.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:50.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:50.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:50.989+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:51.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:51.806+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:51.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:51.952+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:52.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:52.777+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:52.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:52.935+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:53.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:53.779+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:53.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:53.979+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:54.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:54.749+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:54.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:54.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:55.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:55.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:56.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:56.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:56.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:56.723+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:57.055 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:57.055+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:57.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:57.707+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:58.047 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:58.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:58.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:58.681+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:59.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:39:59.007+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:39:59.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:39:59.642+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:00.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:00.043+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:00.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:00.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:01.074 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:01.073+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:01.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:01.602+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:02.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:02.102+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:02.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:02.570+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:03.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:03.133+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:03.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:03.549+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:04.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:04.088+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:04.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:04.551+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:05.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:05.039+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:05.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:05.524+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:06.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:06.083+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:06.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:06.481+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:07.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:07.048+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:07.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:07.504+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:08.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:08.049+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:08.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:08.519+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:09.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:09.031+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:09.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:09.481+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:10.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:10.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:10.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:10.459+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:10.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:10.999+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:11.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:11.441+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:12.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:12.009+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:12.408 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:12.408+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:13.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:13.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:13.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:13.399+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:13.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:13.985+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:14.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:14.390+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:15.015 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:15.014+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:15.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:15.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:16.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:16.006+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:16.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:16.412+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:17.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:17.033+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:17.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:17.391+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:18.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:18.048+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:18.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:18.352+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:19.055 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:19.055+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:19.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:19.364+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:20.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:20.019+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:20.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:20.358+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:21.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:21.035+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:21.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:21.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:22.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:22.081+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:22.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:22.319+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:23.047 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:23.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:23.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:23.365+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:24.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:24.066+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:24.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:24.359+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:25.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:25.037+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:25.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:25.400+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:26.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:26.037+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:26.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:26.421+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:27.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:27.050+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:27.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:27.417+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:28.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:28.038+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:28.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:28.374+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:29.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:29.080+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:29.335 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:29.334+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:30.102 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:30.102+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:30.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:30.358+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:31.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:31.116+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:31.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:31.357+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:32.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:32.152+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:32.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:32.388+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:33.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:33.197+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:33.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:33.371+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:34.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:34.164+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:34.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:34.401+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:35.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:35.127+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:35.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:35.372+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:36.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:36.100+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:36.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:36.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:37.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:37.086+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:37.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:37.341+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:38.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:38.087+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:38.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:38.312+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:39.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:39.057+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:39.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:39.327+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:40.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:40.012+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:40.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:40.328+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:41.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:41.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:41.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:41.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:42.053 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:42.053+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:42.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:42.304+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:43.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:43.081+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:43.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:43.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:44.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:44.099+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:44.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:44.228+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:45.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:45.087+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:45.259 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:45.259+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:46.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:46.078+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:46.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:46.255+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:47.125 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:47.124+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:47.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:47.276+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:48.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:48.165+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:48.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:48.261+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:49.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:49.136+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:49.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:49.234+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:50.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:50.145+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:50.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:50.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:51.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:51.125+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:51.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:51.271+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:52.091 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:52.090+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:52.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:52.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:53.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:53.105+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:53.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:53.312+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:54.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:54.139+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:54.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:54.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:55.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:55.101+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:55.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:55.290+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:56.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:56.141+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:56.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:56.243+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:57.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:57.138+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:57.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:57.260+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:58.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:58.127+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:58.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:58.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:59.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:40:59.159+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:40:59.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:40:59.217+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:00.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:00.171+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:00.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:00.266+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:01.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:01.171+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:01.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:01.271+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:02.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:02.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:02.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:02.275+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:03.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:03.188+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:03.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:03.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:04.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:04.197+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:04.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:04.225+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:05.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:05.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:05.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:05.197+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:06.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:06.211+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:06.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:06.244+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:07.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:07.222+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:07.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:07.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:08.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:08.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:08.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:08.246+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:09.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:09.190+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:09.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:09.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:10.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:10.222+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:10.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:10.305+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:11.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:11.208+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:11.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:11.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:12.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:12.247+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:12.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:12.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:13.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:13.294+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:13.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:13.337+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:14.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:14.343+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:14.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:14.375+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:15.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:15.365+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:15.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:15.385+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:16.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:16.368+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:16.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:16.402+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:17.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:17.373+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:17.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:17.405+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:18.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:18.397+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:18.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:18.401+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:19.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:19.392+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:19.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:19.438+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:20.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:20.429+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:20.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:20.481+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:21.412 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:21.412+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:21.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:21.498+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:22.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:22.447+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:22.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:22.468+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:23.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:23.497+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:23.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:23.518+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:24.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:24.484+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:24.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:24.538+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:25.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:25.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:25.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:25.577+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:26.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:26.479+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:26.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:26.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:27.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:27.481+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:27.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:27.634+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:28.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:28.475+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:28.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:28.669+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:29.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:29.467+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:29.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:29.705+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:30.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:30.502+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:30.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:30.698+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:31.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:31.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:31.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:31.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:32.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:32.510+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:32.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:32.695+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:33.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:33.465+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:33.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:33.732+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:34.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:34.497+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:34.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:34.782+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:35.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:35.493+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:35.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:35.774+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:36.448 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:36.448+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:36.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:36.820+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:37.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:37.445+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:37.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:37.832+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:38.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:38.451+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:38.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:38.809+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:39.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:39.419+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:39.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:39.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:40.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:40.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:40.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:40.789+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:41.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:41.394+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:41.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:41.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:42.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:42.412+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:42.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:42.797+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:43.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:43.449+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:43.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:43.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:44.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:44.423+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:44.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:44.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:45.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:45.451+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:45.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:45.796+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:46.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:46.433+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:46.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:46.801+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:47.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:47.415+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:47.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:47.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:48.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:48.455+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:48.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:48.803+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:49.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:49.463+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:49.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:49.793+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:50.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:50.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:50.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:50.804+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:51.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:51.476+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:51.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:51.790+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:52.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:52.485+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:52.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:52.837+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:53.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:53.521+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:53.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:53.798+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:54.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:54.494+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:54.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:54.824+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:55.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:55.451+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:55.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:55.778+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:56.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:56.418+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:56.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:56.813+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:57.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:57.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:57.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:57.812+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:58.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:58.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:58.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:58.834+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:59.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:41:59.306+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:41:59.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:41:59.868+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:00.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:00.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:00.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:00.886+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:01.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:01.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:01.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:01.881+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:02.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:02.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:02.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:02.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:03.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:03.217+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:03.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:03.836+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:04.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:04.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:04.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:04.842+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:05.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:05.229+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:05.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:05.847+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:06.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:06.226+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:06.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:06.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:07.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:07.209+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:07.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:07.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:08.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:08.224+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:08.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:08.810+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:09.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:09.236+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:09.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:09.802+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:10.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:10.286+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:10.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:10.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:11.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:11.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:11.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:11.770+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:12.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:12.286+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:12.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:12.806+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:13.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:13.332+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:13.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:13.791+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:14.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:14.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:14.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:14.788+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:15.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:15.344+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:15.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:15.775+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:16.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:16.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:16.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:16.728+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:17.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:17.367+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:17.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:17.713+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:18.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:18.385+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:18.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:18.691+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:19.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:19.345+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:19.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:19.694+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:20.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:20.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:20.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:20.734+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:21.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:21.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:21.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:21.778+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:22.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:22.358+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:22.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:22.740+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:23.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:23.408+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:23.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:23.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:24.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:24.380+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:24.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:24.665+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:25.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:25.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:25.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:25.629+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:26.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:26.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:26.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:26.610+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:27.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:27.402+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:27.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:27.601+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:28.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:28.422+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:28.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:28.602+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:29.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:29.464+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:29.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:29.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:30.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:30.505+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:30.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:30.552+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:31.508 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:31.507+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:31.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:31.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:32.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:32.528+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:32.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:32.534+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:33.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:33.511+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:33.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:33.536+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:34.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:34.497+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:34.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:34.520+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:35.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:35.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:35.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:35.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:36.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:36.476+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:36.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:36.486+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:37.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:37.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:37.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:37.534+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:38.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:38.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:38.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:38.540+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:39.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:39.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:39.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:39.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:40.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:40.552+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:40.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:40.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:41.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:41.526+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:41.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:41.604+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:42.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:42.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:42.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:42.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:43.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:43.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:43.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:43.652+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:44.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:44.549+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:44.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:44.621+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:45.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:45.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:45.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:45.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:46.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:46.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:46.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:46.628+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:47.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:47.479+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:47.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:47.664+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:48.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:48.520+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:48.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:48.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:49.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:49.494+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:49.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:49.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:50.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:50.531+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:50.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:50.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:51.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:51.523+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:51.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:51.594+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:52.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:52.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:52.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:52.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:53.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:53.452+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:53.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:53.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:54.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:54.419+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:54.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:54.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:55.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:55.402+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:55.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:55.615+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:56.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:56.399+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:56.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:56.597+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:57.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:57.363+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:57.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:57.593+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:58.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:58.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:58.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:58.558+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:59.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:42:59.352+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:42:59.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:42:59.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:00.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:00.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:00.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:00.567+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:01.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:01.328+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:01.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:01.616+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:02.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:02.354+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:02.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:02.658+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:03.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:03.370+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:03.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:03.621+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:04.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:04.416+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:04.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:04.649+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:05.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:05.399+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:05.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:05.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:06.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:06.422+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:06.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:06.680+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:07.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:07.444+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:07.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:07.656+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:08.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:08.456+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:08.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:08.647+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:09.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:09.479+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:09.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:09.626+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:10.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:10.450+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:10.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:10.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:11.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:11.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:11.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:11.598+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:12.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:12.468+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:12.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:12.570+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:13.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:13.427+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:13.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:13.573+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:14.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:14.426+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:14.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:14.574+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:15.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:15.458+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:15.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:15.537+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:16.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:16.485+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:16.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:16.532+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:17.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:17.520+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:17.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:17.554+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:18.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:18.518+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:18.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:18.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:19.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:19.534+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:19.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:19.606+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:20.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:20.558+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:20.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:20.635+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:21.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:21.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:21.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:21.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:22.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:22.528+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:22.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:22.656+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:23.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:23.510+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:23.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:23.678+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:24.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:24.486+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:24.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:24.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:25.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:25.441+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:25.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:25.706+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:26.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:26.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:26.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:26.659+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:27.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:27.442+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:27.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:27.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:28.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:28.450+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:28.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:28.614+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:29.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:29.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:29.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:29.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:30.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:30.420+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:30.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:30.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:31.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:31.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:31.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:31.620+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:32.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:32.362+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:32.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:32.607+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:33.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:33.336+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:33.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:33.606+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:34.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:34.366+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:34.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:34.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:35.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:35.332+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:35.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:35.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:36.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:36.301+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:36.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:36.592+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:37.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:37.344+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:37.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:37.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:38.355 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:38.356+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:38.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:38.592+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:39.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:39.323+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:39.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:39.569+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:40.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:40.338+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:40.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:40.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:41.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:41.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:41.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:41.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:42.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:42.322+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:42.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:42.552+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:43.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:43.314+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:43.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:43.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:44.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:44.362+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:44.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:44.549+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:45.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:45.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:45.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:45.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:46.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:46.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:46.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:46.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:47.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:47.405+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:47.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:47.560+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:48.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:48.407+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:48.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:48.591+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:49.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:49.371+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:49.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:49.577+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:50.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:50.383+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:50.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:50.570+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:51.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:51.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:51.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:51.572+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:52.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:52.362+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:52.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:52.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:53.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:53.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:53.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:53.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:54.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:54.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:54.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:54.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:55.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:55.366+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:55.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:55.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:56.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:56.368+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:56.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:56.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:57.415 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:57.415+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:57.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:57.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:58.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:58.437+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:58.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:58.516+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:59.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:43:59.407+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:43:59.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:43:59.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:00.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:00.407+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:00.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:00.500+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:01.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:01.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:01.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:01.466+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:02.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:02.409+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:02.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:02.437+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:03.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:03.393+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:03.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:03.394+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:04.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:04.354+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:04.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:04.374+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:05.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:05.384+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:05.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:05.393+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:06.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:06.391+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:06.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:06.398+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:07.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:07.384+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:07.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:07.433+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:08.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:08.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:08.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:08.453+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:09.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:09.421+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:09.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:09.445+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:10.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:10.455+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:10.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:10.464+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:11.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:11.417+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:11.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:11.453+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:12.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:12.441+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:12.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:12.472+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:13.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:13.416+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:13.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:13.475+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:14.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:14.427+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:14.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:14.493+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:15.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:15.381+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:15.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:15.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:16.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:16.386+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:16.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:16.505+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:17.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:17.362+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:17.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:17.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:18.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:18.385+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:18.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:18.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:19.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:19.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:19.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:19.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:20.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:20.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:20.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:20.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:21.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:21.427+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:21.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:21.567+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:22.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:22.396+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:22.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:22.520+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:23.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:23.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:23.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:23.511+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:24.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:24.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:24.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:24.499+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:25.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:25.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:25.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:25.497+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:26.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:26.358+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:26.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:26.523+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:27.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:27.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:27.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:27.553+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:28.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:28.311+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:28.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:28.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:29.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:29.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:29.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:29.531+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:30.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:30.361+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:30.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:30.575+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:31.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:31.337+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:31.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:31.576+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:32.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:32.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:32.599 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:32.599+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:33.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:33.365+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:33.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:33.637+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:34.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:34.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:34.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:34.591+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:35.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:35.355+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:35.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:35.580+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:36.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:36.358+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:36.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:36.611+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:37.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:37.355+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:37.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:37.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:38.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:38.360+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:38.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:38.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:39.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:39.333+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:39.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:39.716+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:40.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:40.315+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:40.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:40.763+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:41.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:41.335+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:41.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:41.813+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:42.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:42.341+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:42.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:42.851+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:43.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:43.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:43.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:43.899+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:44.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:44.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:44.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:44.928+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:45.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:45.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:45.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:45.947+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:46.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:46.319+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:46.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:46.918+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:47.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:47.276+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:47.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:47.897+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:48.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:48.295+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:48.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:48.938+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:49.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:49.337+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:49.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:49.946+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:50.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:50.314+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:50.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:50.944+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:51.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:51.319+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:51.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:51.988+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:52.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:52.351+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:52.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:52.970+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:53.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:53.366+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:53.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:53.974+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:54.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:54.359+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:54.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:54.990+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:55.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:55.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:56.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:56.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:56.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:56.324+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:56.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:56.961+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:57.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:57.303+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:57.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:57.955+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:58.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:58.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:58.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:58.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:59.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:44:59.246+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:44:59.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:44:59.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:00.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:00.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:00.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:00.982+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:01.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:01.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:02.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:02.010+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:02.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:02.239+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:02.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:02.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:03.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:03.240+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:03.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:03.977+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:04.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:04.211+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:04.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:04.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:05.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:05.244+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:05.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:05.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:06.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:06.197+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:06.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:06.952+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:07.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:07.246+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:07.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:07.954+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:08.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:08.236+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:08.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:08.912+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:09.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:09.196+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:09.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:09.866+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:10.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:10.216+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:10.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:10.816+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:11.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:11.240+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:11.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:11.833+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:12.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:12.212+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:12.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:12.818+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:13.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:13.168+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:13.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:13.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:14.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:14.135+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:14.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:14.836+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:15.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:15.121+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:15.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:15.839+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:16.164 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:16.165+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:16.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:16.804+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:17.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:17.188+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:17.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:17.776+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:18.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:18.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:18.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:18.796+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:19.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:19.214+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:19.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:19.750+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:20.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:20.250+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:20.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:20.786+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:21.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:21.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:21.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:21.742+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:22.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:22.157+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:22.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:22.726+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:23.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:23.154+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:23.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:23.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:24.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:24.174+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:24.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:24.694+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:25.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:25.199+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:25.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:25.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:26.183 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:26.184+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:26.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:26.702+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:27.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:27.210+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:27.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:27.726+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:28.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:28.186+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:28.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:28.764+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:29.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:29.167+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:29.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:29.787+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:30.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:30.191+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:30.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:30.811+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:31.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:31.204+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:31.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:31.834+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:32.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:32.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:32.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:32.833+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:33.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:33.263+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:33.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:33.834+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:34.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:34.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:34.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:34.846+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:35.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:35.321+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:35.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:35.830+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:36.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:36.347+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:36.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:36.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:37.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:37.319+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:37.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:37.800+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:38.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:38.341+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:38.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:38.783+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:39.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:39.356+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:39.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:39.767+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:40.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:40.366+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:40.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:40.805+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:41.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:41.351+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:41.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:41.815+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:42.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:42.303+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:42.768 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:42.768+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:43.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:43.258+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:43.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:43.760+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:44.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:44.243+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:44.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:44.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:45.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:45.196+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:45.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:45.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:46.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:46.180+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:46.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:46.744+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:47.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:47.210+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:47.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:47.716+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:48.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:48.197+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:48.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:48.737+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:49.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:49.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:49.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:49.692+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:50.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:50.272+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:50.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:50.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:51.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:51.260+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:51.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:51.683+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:52.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:52.246+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:52.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:52.703+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:53.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:53.267+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:53.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:53.669+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:54.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:54.232+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:54.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:54.628+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:55.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:55.203+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:55.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:55.618+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:56.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:56.214+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:56.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:56.623+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:57.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:57.241+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:57.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:57.593+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:58.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:58.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:58.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:58.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:59.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:45:59.258+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:45:59.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:45:59.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:00.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:00.262+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:00.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:00.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:01.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:01.267+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:01.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:01.680+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:02.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:02.296+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:02.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:02.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:03.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:03.292+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:03.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:03.655+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:04.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:04.250+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:04.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:04.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:05.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:05.272+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:05.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:05.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:06.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:06.248+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:06.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:06.630+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:07.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:07.219+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:07.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:07.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:08.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:08.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:08.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:08.722+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:09.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:09.261+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:09.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:09.741+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:10.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:10.229+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:10.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:10.703+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:11.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:11.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:11.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:11.655+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:12.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:12.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:12.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:12.658+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:13.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:13.238+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:13.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:13.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:14.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:14.215+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:14.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:14.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:15.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:15.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:15.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:15.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:16.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:16.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:16.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:16.668+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:17.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:17.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:17.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:17.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:18.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:18.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:18.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:18.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:19.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:19.284+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:19.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:19.657+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:20.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:20.263+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:20.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:20.634+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:21.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:21.296+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:21.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:21.630+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:22.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:22.316+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:22.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:22.604+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:23.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:23.314+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:23.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:23.584+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:24.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:24.281+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:24.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:24.590+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:25.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:25.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:25.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:25.598+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:26.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:26.320+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:26.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:26.568+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:27.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:27.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:27.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:27.574+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:28.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:28.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:28.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:28.578+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:29.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:29.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:29.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:29.575+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:30.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:30.308+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:30.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:30.584+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:31.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:31.321+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:31.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:31.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:32.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:32.286+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:32.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:32.589+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:33.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:33.270+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:33.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:33.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:34.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:34.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:34.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:34.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:35.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:35.212+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:35.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:35.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:36.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:36.194+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:36.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:36.532+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:37.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:37.237+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:37.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:37.578+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:38.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:38.262+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:38.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:38.626+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:39.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:39.268+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:39.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:39.664+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:40.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:40.314+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:40.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:40.624+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:41.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:41.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:41.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:41.661+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:42.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:42.276+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:42.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:42.675+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:43.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:43.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:43.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:43.671+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:44.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:44.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:44.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:44.698+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:45.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:45.266+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:45.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:45.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:46.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:46.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:46.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:46.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:47.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:47.288+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:47.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:47.702+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:48.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:48.304+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:48.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:48.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:49.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:49.313+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:49.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:49.630+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:50.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:50.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:50.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:50.587+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:51.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:51.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:51.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:51.578+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:52.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:52.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:52.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:52.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:53.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:53.289+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:53.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:53.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:54.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:54.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:54.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:54.560+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:55.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:55.277+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:55.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:55.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:56.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:56.288+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:56.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:56.511+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:57.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:57.293+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:57.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:57.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:58.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:58.289+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:58.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:58.541+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:59.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:46:59.272+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:46:59.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:46:59.528+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:00.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:00.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:00.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:00.484+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:01.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:01.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:01.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:01.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:02.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:02.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:02.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:02.438+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:03.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:03.240+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:03.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:03.398+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:04.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:04.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:04.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:04.385+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:05.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:05.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:05.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:05.347+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:06.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:06.221+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:06.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:06.305+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:07.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:07.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:07.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:07.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:08.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:08.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:08.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:08.255+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:09.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:09.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:09.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:09.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:10.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:10.225+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:10.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:10.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:11.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:11.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:11.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:11.349+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:12.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:12.192+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:12.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:12.333+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:13.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:13.242+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:13.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:13.311+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:14.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:14.277+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:14.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:14.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:15.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:15.240+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:15.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:15.254+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:16.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:16.215+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:16.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:16.275+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:17.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:17.174+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:17.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:17.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:18.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:18.203+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:18.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:18.304+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:19.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:19.195+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:19.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:19.285+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:20.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:20.191+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:20.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:20.240+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:21.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:21.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:21.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:21.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:22.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:22.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:22.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:22.285+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:23.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:23.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:23.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:23.264+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:24.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:24.218+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:24.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:24.231+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:25.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:25.205+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:25.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:25.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:26.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:26.223+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:26.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:26.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:27.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:27.261+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:27.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:27.293+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:28.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:28.262+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:28.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:28.268+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:29.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:29.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:29.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:29.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:30.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:30.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:30.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:30.311+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:31.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:31.276+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:31.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:31.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:32.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:32.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:32.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:32.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:33.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:33.257+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:33.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:33.283+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:34.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:34.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:34.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:34.314+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:35.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:35.203+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:35.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:35.319+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:36.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:36.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:36.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:36.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:37.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:37.187+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:37.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:37.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:38.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:38.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:38.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:38.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:39.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:39.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:39.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:39.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:40.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:40.095+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:40.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:40.242+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:41.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:41.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:41.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:41.193+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:42.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:42.115+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:42.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:42.236+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:43.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:43.119+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:43.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:43.219+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:44.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:44.102+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:44.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:44.193+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:45.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:45.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:45.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:45.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:46.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:46.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:46.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:46.229+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:47.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:47.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:47.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:47.218+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:48.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:48.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:48.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:48.266+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:49.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:49.126+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:49.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:49.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:50.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:50.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:50.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:50.302+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:51.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:51.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:51.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:51.259+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:52.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:52.106+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:52.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:52.264+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:53.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:53.084+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:53.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:53.260+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:54.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:54.074+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:54.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:54.237+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:55.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:55.044+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:55.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:55.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:56.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:56.076+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:56.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:56.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:57.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:57.108+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:57.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:57.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:58.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:58.117+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:58.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:58.298+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:59.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:47:59.156+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:47:59.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:47:59.337+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:00.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:00.185+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:00.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:00.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:01.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:01.225+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:01.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:01.322+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:02.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:02.248+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:02.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:02.296+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:03.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:03.235+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:03.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:03.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:04.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:04.266+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:04.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:04.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:05.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:05.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:05.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:05.306+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:06.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:06.240+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:06.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:06.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:07.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:07.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:07.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:07.302+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:08.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:08.243+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:08.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:08.306+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:09.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:09.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:09.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:09.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:10.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:10.225+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:10.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:10.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:11.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:11.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:11.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:11.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:12.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:12.248+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:12.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:12.310+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:13.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:13.218+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:13.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:13.336+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:14.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:14.254+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:14.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:14.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:15.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:15.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:15.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:15.380+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:16.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:16.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:16.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:16.421+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:17.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:17.191+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:17.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:17.418+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:18.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:18.178+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:18.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:18.376+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:19.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:19.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:19.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:19.342+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:20.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:20.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:20.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:20.390+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:21.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:21.155+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:21.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:21.423+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:22.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:22.177+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:22.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:22.468+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:23.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:23.181+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:23.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:23.442+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:24.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:24.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:24.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:24.478+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:25.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:25.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:25.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:25.450+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:26.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:26.065+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:26.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:26.407+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:27.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:27.106+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:27.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:27.388+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:28.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:28.132+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:28.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:28.350+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:29.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:29.172+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:29.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:29.340+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:30.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:30.142+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:30.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:30.328+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:31.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:31.166+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:31.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:31.368+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:32.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:32.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:32.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:32.398+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:33.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:33.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:33.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:33.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:34.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:34.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:34.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:34.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:35.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:35.106+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:35.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:35.430+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:36.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:36.072+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:36.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:36.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:37.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:37.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:37.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:37.442+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:38.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:38.150+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:38.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:38.422+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:39.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:39.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:39.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:39.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:40.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:40.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:40.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:40.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:41.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:41.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:41.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:41.499+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:42.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:42.158+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:42.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:42.511+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:43.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:43.130+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:43.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:43.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:44.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:44.094+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:44.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:44.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:45.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:45.136+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:45.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:45.416+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:46.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:46.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:46.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:46.423+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:47.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:47.108+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:47.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:47.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:48.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:48.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:48.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:48.409+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:49.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:49.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:49.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:49.396+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:50.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:50.099+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:50.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:50.372+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:51.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:51.057+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:51.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:51.365+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:52.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:52.105+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:52.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:52.359+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:53.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:53.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:53.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:53.340+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:54.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:54.143+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:54.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:54.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:55.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:55.153+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:55.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:55.362+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:56.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:56.171+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:56.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:56.338+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:57.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:57.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:57.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:57.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:58.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:58.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:58.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:58.352+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:59.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:48:59.105+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:48:59.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:48:59.348+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:00.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:00.125+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:00.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:00.391+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:01.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:01.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:01.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:01.435+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:02.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:02.177+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:02.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:02.417+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:03.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:03.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:03.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:03.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:04.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:04.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:04.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:04.399+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:05.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:05.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:05.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:05.409+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:06.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:06.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:06.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:06.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:07.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:07.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:07.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:07.390+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:08.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:08.125+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:08.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:08.360+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:09.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:09.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:09.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:09.373+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:10.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:10.077+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:10.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:10.374+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:11.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:11.128+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:11.364 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:11.365+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:12.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:12.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:12.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:12.380+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:13.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:13.175+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:13.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:13.410+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:14.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:14.128+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:14.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:14.416+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:15.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:15.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:15.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:15.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:16.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:16.115+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:16.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:16.398+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:17.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:17.097+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:17.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:17.443+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:18.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:18.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:18.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:18.457+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:19.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:19.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:19.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:19.456+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:20.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:20.117+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:20.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:20.451+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:21.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:21.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:21.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:21.460+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:22.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:22.079+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:22.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:22.471+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:23.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:23.042+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:23.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:23.464+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:24.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:24.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:24.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:24.450+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:25.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:25.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:25.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:25.475+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:26.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:26.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:26.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:26.515+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:27.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:27.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:27.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:27.515+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:28.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:28.115+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:28.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:28.475+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:29.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:29.155+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:29.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:29.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:30.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:30.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:30.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:30.440+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:31.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:31.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:31.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:31.443+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:32.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:32.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:32.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:32.418+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:33.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:33.195+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:33.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:33.435+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:34.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:34.158+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:34.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:34.389+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:35.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:35.195+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:35.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:35.368+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:36.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:36.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:36.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:36.397+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:37.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:37.112+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:37.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:37.377+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:38.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:38.126+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:38.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:38.407+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:39.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:39.157+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:39.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:39.429+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:40.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:40.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:40.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:40.456+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:41.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:41.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:41.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:41.432+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:42.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:42.185+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:42.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:42.459+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:43.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:43.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:43.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:43.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:44.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:44.177+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:44.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:44.531+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:45.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:45.142+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:45.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:45.570+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:46.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:46.131+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:46.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:46.530+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:47.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:47.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:47.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:47.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:48.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:48.180+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:48.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:48.480+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:49.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:49.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:49.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:49.506+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:50.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:50.230+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:50.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:50.478+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:51.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:51.248+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:51.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:51.485+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:52.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:52.228+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:52.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:52.478+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:53.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:53.213+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:53.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:53.504+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:54.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:54.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:54.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:54.550+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:55.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:55.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:55.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:55.582+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:56.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:56.201+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:56.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:56.579+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:57.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:57.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:57.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:57.574+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:58.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:58.164+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:58.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:58.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:59.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:49:59.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:49:59.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:49:59.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:00.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:00.172+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:00.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:00.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:01.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:01.152+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:01.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:01.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:02.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:02.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:02.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:02.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:03.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:03.106+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:03.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:03.696+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:04.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:04.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:04.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:04.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:05.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:05.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:05.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:05.633+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:06.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:06.128+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:06.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:06.632+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:07.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:07.122+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:07.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:07.656+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:08.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:08.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:08.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:08.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:09.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:09.187+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:09.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:09.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:10.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:10.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:10.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:10.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:11.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:11.150+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:11.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:11.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:12.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:12.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:12.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:12.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:13.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:13.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:13.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:13.686+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:14.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:14.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:14.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:14.686+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:15.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:15.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:15.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:15.674+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:16.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:16.168+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:16.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:16.650+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:17.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:17.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:17.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:17.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:18.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:18.112+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:18.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:18.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:19.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:19.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:19.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:19.563+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:20.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:20.186+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:20.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:20.561+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:21.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:21.234+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:21.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:21.575+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:22.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:22.260+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:22.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:22.534+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:23.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:23.276+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:23.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:23.563+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:24.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:24.286+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:24.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:24.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:25.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:25.239+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:25.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:25.542+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:26.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:26.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:26.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:26.565+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:27.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:27.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:27.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:27.523+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:28.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:28.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:28.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:28.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:29.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:29.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:29.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:29.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:30.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:30.099+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:30.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:30.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:31.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:31.061+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:31.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:31.562+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:32.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:32.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:32.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:32.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:33.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:33.009+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:33.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:33.588+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:34.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:34.032+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:34.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:34.601+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:35.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:35.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:35.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:35.596+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:36.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:36.040+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:36.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:36.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:37.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:37.026+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:37.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:37.561+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:38.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:38.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:38.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:38.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:39.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:39.097+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:39.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:39.528+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:40.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:40.063+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:40.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:40.544+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:41.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:41.047+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:41.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:41.525+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:42.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:42.055+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:42.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:42.499+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:43.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:43.093+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:43.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:43.516+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:44.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:44.100+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:44.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:44.547+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:45.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:45.077+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:45.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:45.502+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:46.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:46.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:46.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:46.546+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:47.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:47.037+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:47.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:47.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:48.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:48.044+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:48.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:48.570+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:49.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:49.074+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:49.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:49.592+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:50.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:50.111+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:50.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:50.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:51.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:51.100+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:51.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:51.533+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:52.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:52.123+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:52.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:52.540+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:53.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:53.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:53.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:53.553+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:54.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:54.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:54.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:54.514+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:55.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:55.156+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:55.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:55.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:56.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:56.154+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:56.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:56.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:57.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:57.168+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:57.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:57.514+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:58.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:58.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:58.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:58.548+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:59.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:50:59.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:50:59.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:50:59.590+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:00.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:00.154+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:00.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:00.593+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:01.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:01.199+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:01.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:01.584+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:02.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:02.233+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:02.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:02.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:03.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:03.284+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:03.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:03.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:04.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:04.258+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:04.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:04.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:05.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:05.270+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:05.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:05.596+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:06.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:06.222+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:06.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:06.621+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:07.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:07.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:07.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:07.586+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:08.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:08.212+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:08.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:08.605+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:09.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:09.260+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:09.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:09.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:10.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:10.219+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:10.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:10.593+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:11.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:11.189+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:11.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:11.559+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:12.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:12.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:12.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:12.570+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:13.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:13.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:13.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:13.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:14.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:14.228+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:14.491 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:14.492+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:15.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:15.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:15.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:15.511+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:16.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:16.266+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:16.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:16.540+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:17.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:17.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:17.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:17.581+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:18.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:18.256+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:18.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:18.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:19.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:19.242+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:19.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:19.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:20.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:20.205+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:20.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:20.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:21.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:21.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:21.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:21.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:22.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:22.246+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:22.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:22.590+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:23.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:23.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:23.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:23.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:24.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:24.221+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:24.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:24.594+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:25.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:25.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:25.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:25.623+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:26.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:26.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:26.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:26.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:27.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:27.130+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:27.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:27.622+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:28.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:28.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:28.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:28.583+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:29.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:29.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:29.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:29.557+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:30.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:30.196+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:30.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:30.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:31.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:31.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:31.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:31.532+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:32.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:32.188+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:32.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:32.544+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:33.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:33.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:33.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:33.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:34.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:34.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:34.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:34.536+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:35.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:35.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:35.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:35.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:36.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:36.232+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:36.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:36.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:37.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:37.257+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:37.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:37.626+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:38.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:38.236+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:38.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:38.600+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:39.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:39.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:39.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:39.646+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:40.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:40.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:40.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:40.619+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:41.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:41.238+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:41.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:41.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:42.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:42.246+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:42.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:42.593+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:43.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:43.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:43.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:43.575+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:44.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:44.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:44.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:44.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:45.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:45.291+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:45.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:45.528+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:46.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:46.332+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:46.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:46.571+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:47.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:47.357+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:47.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:47.561+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:48.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:48.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:48.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:48.535+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:49.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:49.394+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:49.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:49.542+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:50.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:50.390+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:50.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:50.582+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:51.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:51.421+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:51.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:51.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:52.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:52.465+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:52.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:52.594+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:53.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:53.430+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:53.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:53.567+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:54.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:54.419+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:54.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:54.539+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:55.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:55.419+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:55.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:55.589+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:56.415 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:56.415+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:56.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:56.571+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:57.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:57.446+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:57.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:57.607+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:58.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:58.461+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:58.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:58.570+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:59.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:51:59.474+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:51:59.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:51:59.583+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:00.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:00.505+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:00.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:00.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:01.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:01.471+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:01.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:01.646+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:02.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:02.484+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:02.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:02.683+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:03.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:03.482+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:03.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:03.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:04.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:04.482+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:04.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:04.665+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:05.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:05.481+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:05.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:05.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:06.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:06.507+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:06.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:06.670+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:07.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:07.469+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:07.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:07.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:08.495 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:08.495+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:08.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:08.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:09.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:09.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:09.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:09.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:10.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:10.474+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:10.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:10.665+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:11.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:11.482+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:11.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:11.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:12.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:12.523+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:12.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:12.645+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:13.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:13.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:13.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:13.691+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:14.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:14.513+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:14.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:14.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:15.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:15.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:15.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:15.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:16.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:16.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:16.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:16.695+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:17.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:17.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:17.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:17.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:18.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:18.589+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:18.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:18.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:19.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:19.639+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:19.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:19.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:20.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:20.624+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:20.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:20.721+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:21.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:21.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:21.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:21.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:22.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:22.643+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:22.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:22.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:23.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:23.671+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:23.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:23.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:24.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:24.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:24.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:24.793+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:25.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:25.667+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:25.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:25.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:26.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:26.669+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:26.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:26.817+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:27.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:27.655+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:27.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:27.863+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:28.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:28.651+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:28.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:28.855+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:29.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:29.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:29.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:29.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:30.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:30.701+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:30.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:30.813+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:31.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:31.681+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:31.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:31.851+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:32.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:32.657+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:32.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:32.845+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:33.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:33.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:33.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:33.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:34.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:34.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:34.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:34.851+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:35.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:35.711+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:35.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:35.840+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:36.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:36.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:36.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:36.792+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:37.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:37.778+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:37.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:37.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:38.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:38.767+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:38.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:38.783+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:39.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:39.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:39.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:39.759+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:40.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:40.785+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:40.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:40.811+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:41.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:41.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:41.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:41.845+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:42.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:42.818+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:42.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:42.825+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:43.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:43.804+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:43.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:43.824+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:44.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:44.810+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:44.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:44.868+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:45.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:45.779+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:45.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:45.882+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:46.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:46.770+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:46.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:46.930+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:47.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:47.737+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:47.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:47.908+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:48.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:48.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:48.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:48.913+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:49.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:49.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:49.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:49.869+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:50.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:50.665+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:50.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:50.906+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:51.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:51.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:51.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:51.954+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:52.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:52.709+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:52.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:52.930+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:53.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:53.746+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:53.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:53.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:54.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:54.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:54.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:54.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:55.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:55.821+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:55.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:55.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:56.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:56.824+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:56.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:56.890+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:57.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:57.809+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:57.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:57.911+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:58.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:58.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:58.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:58.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:59.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:52:59.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:52:59.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:52:59.928+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:00.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:00.814+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:00.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:00.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:01.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:01.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:01.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:01.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:02.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:02.821+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:02.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:02.924+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:03.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:03.791+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:03.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:03.918+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:04.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:04.784+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:04.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:04.898+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:05.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:05.790+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:05.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:05.898+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:06.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:06.838+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:06.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:06.904+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:07.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:07.880+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:07.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:07.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:08.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:08.902+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:08.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:08.930+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:09.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:09.892+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:09.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:09.928+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:10.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:10.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:10.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:10.944+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:11.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:11.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:11.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:11.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:12.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:12.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:12.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:13.000+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:13.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:13.817+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:14.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:14.019+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:14.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:14.799+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:15.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:15.024+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:15.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:15.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:16.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:16.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:16.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:16.732+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:17.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:17.026+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:17.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:17.766+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:17.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:17.986+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:18.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:18.782+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:18.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:18.986+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:19.764 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:19.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:19.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:19.997+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:20.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:20.757+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:21.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:21.029+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:21.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:21.800+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:22.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:22.005+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:22.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:22.766+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:22.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:22.983+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:23.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:23.774+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:23.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:23.952+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:24.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:24.802+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:24.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:24.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:25.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:25.817+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:25.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:25.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:26.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:26.783+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:26.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:26.977+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:27.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:27.803+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:27.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:27.966+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:28.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:28.755+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:29.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:29.006+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:29.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:29.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:30.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:30.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:30.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:30.736+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:30.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:30.984+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:31.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:31.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:31.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:31.950+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:32.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:32.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:32.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:32.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:33.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:33.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:33.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:33.944+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:34.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:34.652+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:34.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:34.902+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:35.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:35.680+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:35.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:35.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:36.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:36.730+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:36.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:36.857+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:37.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:37.751+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:37.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:37.867+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:38.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:38.761+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:38.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:38.885+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:39.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:39.744+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:39.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:39.877+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:40.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:40.738+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:40.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:40.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:41.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:41.703+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:41.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:41.837+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:42.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:42.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:42.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:42.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:43.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:43.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:43.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:43.865+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:44.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:44.775+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:44.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:44.853+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:45.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:45.812+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:45.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:45.877+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:46.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:46.779+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:46.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:46.861+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:47.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:47.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:47.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:47.907+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:48.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:48.734+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:48.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:48.859+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:49.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:49.745+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:49.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:49.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:50.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:50.711+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:50.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:50.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:51.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:51.719+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:51.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:51.777+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:52.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:52.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:52.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:52.765+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:53.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:53.714+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:53.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:53.774+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:54.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:54.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:54.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:54.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:55.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:55.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:55.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:55.793+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:56.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:56.710+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:56.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:56.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:57.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:57.703+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:57.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:57.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:58.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:58.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:58.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:58.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:59.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:53:59.698+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:53:59.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:53:59.772+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:00.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:00.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:00.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:00.757+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:01.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:01.710+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:01.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:01.733+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:02.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:02.757+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:02.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:02.769+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:03.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:03.766+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:03.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:03.769+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:04.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:04.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:04.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:04.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:05.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:05.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:05.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:05.765+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:06.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:06.645+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:06.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:06.726+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:07.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:07.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:07.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:07.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:08.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:08.677+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:08.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:08.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:09.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:09.716+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:09.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:09.724+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:10.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:10.693+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:10.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:10.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:11.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:11.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:11.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:11.680+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:12.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:12.653+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:12.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:12.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:13.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:13.641+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:13.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:13.700+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:14.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:14.641+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:14.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:14.747+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:15.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:15.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:15.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:15.789+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:16.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:16.591+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:16.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:16.779+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:17.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:17.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:17.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:17.751+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:18.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:18.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:18.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:18.722+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:19.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:19.604+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:19.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:19.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:20.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:20.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:20.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:20.695+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:21.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:21.610+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:21.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:21.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:22.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:22.573+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:22.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:22.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:23.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:23.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:23.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:23.684+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:24.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:24.557+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:24.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:24.721+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:25.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:25.562+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:25.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:25.752+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:26.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:26.533+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:26.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:26.735+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:27.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:27.507+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:27.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:27.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:28.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:28.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:28.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:28.708+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:29.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:29.568+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:29.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:29.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:30.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:30.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:30.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:30.669+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:31.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:31.567+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:31.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:31.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:32.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:32.559+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:32.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:32.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:33.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:33.557+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:33.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:33.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:34.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:34.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:34.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:34.776+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:35.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:35.541+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:35.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:35.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:36.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:36.524+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:36.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:36.767+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:37.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:37.549+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:37.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:37.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:38.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:38.521+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:38.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:38.710+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:39.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:39.552+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:39.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:39.675+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:40.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:40.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:40.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:40.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:41.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:41.535+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:41.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:41.674+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:42.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:42.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:42.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:42.686+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:43.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:43.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:43.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:43.704+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:44.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:44.604+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:44.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:44.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:45.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:45.605+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:45.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:45.735+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:46.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:46.612+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:46.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:46.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:47.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:47.613+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:47.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:47.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:48.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:48.573+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:48.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:48.727+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:49.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:49.575+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:49.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:49.763+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:50.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:50.586+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:50.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:50.722+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:51.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:51.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:51.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:51.753+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:52.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:52.585+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:52.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:52.800+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:53.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:53.623+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:53.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:53.753+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:54.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:54.577+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:54.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:54.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:55.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:55.595+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:55.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:55.777+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:56.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:56.555+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:56.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:56.803+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:57.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:57.562+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:57.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:57.805+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:58.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:58.581+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:58.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:58.811+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:59.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:54:59.536+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:54:59.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:54:59.816+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:00.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:00.537+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:00.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:00.847+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:01.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:01.532+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:01.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:01.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:02.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:02.559+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:02.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:02.819+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:03.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:03.589+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:03.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:03.835+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:04.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:04.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:04.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:04.851+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:05.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:05.577+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:05.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:05.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:06.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:06.556+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:06.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:06.911+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:07.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:07.536+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:07.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:07.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:08.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:08.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:08.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:08.933+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:09.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:09.557+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:09.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:09.929+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:10.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:10.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:10.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:10.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:11.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:11.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:11.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:11.957+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:12.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:12.646+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:12.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:12.990+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:13.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:13.648+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:13.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:13.947+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:14.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:14.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:14.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:14.976+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:15.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:15.595+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:15.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:15.954+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:16.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:16.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:16.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:16.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:17.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:17.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:17.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:17.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:18.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:18.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:18.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:18.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:19.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:19.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:19.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:19.917+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:20.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:20.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:20.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:20.963+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:21.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:21.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:21.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:21.977+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:22.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:22.629+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:22.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:22.965+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:23.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:23.641+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:23.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:23.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:24.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:24.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:24.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:24.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:25.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:25.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:25.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:25.901+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:26.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:26.679+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:26.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:26.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:27.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:27.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:27.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:27.897+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:28.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:28.683+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:28.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:28.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:29.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:29.644+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:29.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:29.849+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:30.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:30.639+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:30.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:30.803+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:31.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:31.645+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:31.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:31.787+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:32.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:32.693+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:32.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:32.829+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:33.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:33.668+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:33.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:33.800+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:34.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:34.659+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:34.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:34.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:35.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:35.671+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:35.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:35.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:36.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:36.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:36.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:36.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:37.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:37.719+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:37.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:37.868+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:38.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:38.713+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:38.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:38.836+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:39.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:39.733+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:39.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:39.844+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:40.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:40.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:40.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:40.882+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:41.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:41.716+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:41.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:41.918+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:42.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:42.702+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:42.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:42.942+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:43.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:43.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:43.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:43.983+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:44.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:44.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:44.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:44.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:45.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:45.682+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:45.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:45.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:46.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:46.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:46.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:46.933+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:47.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:47.705+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:47.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:47.966+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:48.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:48.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:48.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:48.963+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:49.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:49.730+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:49.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:49.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:50.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:50.748+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:50.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:50.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:51.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:51.734+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:51.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:51.940+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:52.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:52.698+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:52.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:52.947+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:53.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:53.733+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:53.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:53.993+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:54.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:54.775+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:55.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:55.034+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:55.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:55.773+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:56.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:56.084+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:56.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:56.784+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:57.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:57.117+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:57.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:57.770+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:58.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:58.115+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:58.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:58.748+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:59.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:55:59.071+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:55:59.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:55:59.740+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:00.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:00.075+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:00.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:00.736+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:01.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:01.102+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:01.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:01.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:02.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:02.077+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:02.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:02.688+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:03.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:03.105+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:03.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:03.650+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:04.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:04.082+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:04.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:04.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:05.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:05.103+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:05.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:05.710+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:06.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:06.099+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:06.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:06.749+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:07.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:07.062+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:07.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:07.717+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:08.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:08.072+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:08.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:08.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:09.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:09.084+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:09.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:09.667+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:10.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:10.099+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:10.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:10.661+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:11.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:11.103+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:11.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:11.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:12.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:12.075+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:12.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:12.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:13.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:13.081+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:13.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:13.743+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:14.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:14.109+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:14.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:14.782+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:15.132 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:15.131+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:15.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:15.787+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:16.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:16.162+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:16.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:16.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:17.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:17.172+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:17.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:17.783+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:18.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:18.126+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:18.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:18.822+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:19.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:19.115+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:19.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:19.845+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:20.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:20.087+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:20.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:20.878+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:21.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:21.049+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:21.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:21.911+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:22.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:22.098+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:22.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:22.863+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:23.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:23.086+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:23.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:23.875+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:24.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:24.037+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:24.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:24.863+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:25.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:25.031+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:25.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:25.847+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:26.064 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:26.063+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:26.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:26.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:27.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:27.096+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:27.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:27.884+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:28.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:28.116+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:28.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:28.883+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:29.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:29.157+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:29.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:29.872+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:30.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:30.183+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:30.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:30.839+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:31.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:31.166+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:31.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:31.857+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:32.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:32.187+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:32.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:32.831+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:33.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:33.170+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:33.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:33.821+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:34.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:34.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:34.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:34.826+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:35.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:35.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:35.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:35.847+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:36.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:36.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:36.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:36.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:37.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:37.292+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:37.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:37.841+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:38.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:38.304+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:38.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:38.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:39.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:39.325+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:39.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:39.817+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:40.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:40.281+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:40.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:40.811+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:41.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:41.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:41.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:41.784+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:42.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:42.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:42.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:42.814+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:43.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:43.213+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:43.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:43.851+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:44.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:44.244+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:44.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:44.893+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:45.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:45.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:45.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:45.867+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:46.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:46.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:46.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:46.916+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:47.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:47.211+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:47.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:47.887+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:48.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:48.246+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:48.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:48.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:49.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:49.243+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:49.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:49.861+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:50.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:50.270+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:50.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:50.885+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:51.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:51.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:51.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:51.900+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:52.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:52.256+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:52.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:52.946+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:53.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:53.281+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:53.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:53.943+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:54.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:54.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:54.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:54.914+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:55.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:55.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:55.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:55.872+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:56.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:56.311+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:56.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:56.837+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:57.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:57.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:57.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:57.797+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:58.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:58.306+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:58.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:58.758+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:59.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:56:59.313+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:56:59.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:56:59.793+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:00.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:00.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:00.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:00.781+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:01.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:01.349+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:01.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:01.823+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:02.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:02.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:02.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:02.811+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:03.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:03.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:03.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:03.777+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:04.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:04.356+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:04.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:04.737+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:05.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:05.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:05.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:05.717+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:06.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:06.428+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:06.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:06.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:07.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:07.424+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:07.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:07.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:08.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:08.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:08.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:08.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:09.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:09.437+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:09.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:09.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:10.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:10.481+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:10.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:10.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:11.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:11.480+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:11.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:11.659+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:12.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:12.518+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:12.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:12.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:13.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:13.552+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:13.688 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:13.688+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:14.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:14.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:14.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:14.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:15.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:15.599+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:15.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:15.714+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:16.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:16.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:16.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:16.679+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:17.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:17.598+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:17.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:17.661+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:18.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:18.565+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:18.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:18.629+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:19.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:19.591+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:19.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:19.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:20.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:20.586+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:20.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:20.643+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:21.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:21.543+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:21.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:21.650+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:22.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:22.533+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:22.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:22.610+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:23.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:23.565+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:23.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:23.613+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:24.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:24.603+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:24.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:24.608+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:25.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:25.597+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:25.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:25.633+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:26.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:26.565+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:26.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:26.625+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:27.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:27.549+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:27.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:27.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:28.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:28.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:28.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:28.562+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:29.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:29.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:29.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:29.603+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:30.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:30.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:30.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:30.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:31.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:31.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:31.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:31.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:32.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:32.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:32.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:32.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:33.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:33.521+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:33.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:33.530+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:34.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:34.500+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:34.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:34.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:35.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:35.464+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:35.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:35.534+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:36.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:36.504+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:36.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:36.520+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:37.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:37.505+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:37.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:37.540+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:38.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:38.468+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:38.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:38.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:39.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:39.434+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:39.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:39.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:40.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:40.478+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:40.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:40.547+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:41.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:41.486+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:41.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:41.529+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:42.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:42.494+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:42.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:42.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:43.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:43.452+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:43.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:43.511+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:44.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:44.428+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:44.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:44.471+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:45.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:45.421+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:45.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:45.440+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:46.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:46.377+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:46.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:46.472+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:47.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:47.386+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:47.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:47.502+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:48.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:48.351+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:48.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:48.496+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:49.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:49.385+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:49.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:49.449+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:50.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:50.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:50.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:50.435+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:51.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:51.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:51.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:51.436+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:52.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:52.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:52.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:52.414+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:53.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:53.284+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:53.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:53.397+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:54.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:54.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:54.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:54.446+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:55.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:55.343+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:55.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:55.442+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:56.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:56.297+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:56.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:56.472+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:57.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:57.283+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:57.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:57.520+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:58.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:58.304+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:58.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:58.543+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:59.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:57:59.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:57:59.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:57:59.506+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:00.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:00.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:00.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:00.532+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:01.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:01.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:01.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:01.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:02.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:02.372+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:02.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:02.565+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:03.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:03.332+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:03.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:03.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:04.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:04.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:04.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:04.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:05.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:05.369+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:05.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:05.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:06.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:06.367+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:06.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:06.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:07.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:07.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:07.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:07.582+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:08.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:08.449+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:08.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:08.613+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:09.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:09.456+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:09.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:09.606+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:10.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:10.463+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:10.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:10.583+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:11.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:11.451+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:11.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:11.620+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:12.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:12.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:12.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:12.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:13.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:13.538+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:13.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:13.606+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:14.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:14.498+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:14.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:14.645+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:15.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:15.502+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:15.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:15.694+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:16.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:16.463+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:16.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:16.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:17.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:17.484+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:17.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:17.735+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:18.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:18.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:18.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:18.728+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:19.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:19.523+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:19.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:19.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:20.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:20.526+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:20.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:20.668+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:21.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:21.557+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:21.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:21.662+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:22.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:22.540+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:22.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:22.695+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:23.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:23.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:23.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:23.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:24.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:24.543+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:24.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:24.691+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:25.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:25.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:25.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:25.658+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:26.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:26.536+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:26.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:26.638+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:27.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:27.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:27.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:27.681+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:28.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:28.585+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:28.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:28.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:29.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:29.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:29.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:29.693+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:30.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:30.590+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:30.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:30.678+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:31.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:31.631+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:31.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:31.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:32.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:32.628+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:32.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:32.650+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:33.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:33.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:33.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:33.658+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:34.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:34.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:34.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:34.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:35.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:35.664+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:35.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:35.679+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:36.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:36.664+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:36.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:36.683+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:37.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:37.639+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:37.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:37.721+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:38.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:38.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:38.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:38.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:39.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:39.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:39.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:39.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:40.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:40.635+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:40.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:40.707+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:41.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:41.637+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:41.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:41.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:42.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:42.655+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:42.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:42.724+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:43.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:43.651+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:43.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:43.695+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:44.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:44.637+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:44.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:44.709+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:45.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:45.680+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:45.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:45.714+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:46.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:46.715+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:46.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:46.719+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:47.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:47.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:47.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:47.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:48.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:48.717+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:48.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:48.719+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:49.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:49.704+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:49.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:49.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:50.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:50.731+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:50.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:50.742+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:51.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:51.767+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:51.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:51.779+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:52.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:52.730+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:52.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:52.738+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:53.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:53.691+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:53.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:53.753+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:54.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:54.702+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:54.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:54.730+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:55.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:55.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:55.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:55.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:56.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:56.728+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:56.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:56.757+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:57.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:57.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:57.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:57.775+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:58.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:58.756+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:58.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:58.758+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:59.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:58:59.714+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:58:59.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:58:59.716+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:00.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:00.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:00.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:00.739+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:01.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:01.691+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:01.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:01.732+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:02.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:02.727+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:02.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:02.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:03.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:03.710+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:03.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:03.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:04.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:04.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:04.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:04.762+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:05.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:05.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:05.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:05.725+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:06.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:06.733+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:06.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:06.762+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:07.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:07.731+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:07.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:07.746+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:08.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:08.705+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:08.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:08.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:09.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:09.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:09.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:09.746+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:10.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:10.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:10.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:10.791+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:11.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:11.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:11.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:11.781+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:12.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:12.703+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:12.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:12.830+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:13.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:13.751+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:13.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:13.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:14.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:14.719+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:14.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:14.899+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:15.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:15.682+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:15.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:15.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:16.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:16.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:16.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:16.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:17.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:17.713+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:17.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:17.855+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:18.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:18.752+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:18.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:18.813+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:19.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:19.707+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:19.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:19.822+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:20.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:20.712+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:20.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:20.861+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:21.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:21.749+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:21.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:21.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:22.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:22.749+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:22.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:22.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:23.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:23.753+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:23.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:23.911+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:24.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:24.729+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:24.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:24.908+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:25.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:25.768+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:25.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:25.883+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:26.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:26.770+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:26.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:26.932+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:27.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:27.730+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:27.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:27.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:28.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:28.757+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:28.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:28.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:29.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:29.754+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:29.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:29.943+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:30.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:30.749+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:30.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:30.930+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:31.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:31.702+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:31.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:31.925+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:32.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:32.715+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:32.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:32.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:33.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:33.762+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:33.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:33.902+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:34.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:34.799+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:34.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:34.944+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:35.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:35.808+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:35.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:35.951+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:36.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:36.777+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:36.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:36.981+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:37.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:37.775+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:37.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:37.949+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:38.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:38.808+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:38.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:38.969+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:39.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:39.800+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:40.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:40.006+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:40.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:40.791+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:41.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:41.054+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:41.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:41.764+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:42.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:42.073+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:42.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:42.762+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:43.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:43.102+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:43.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:43.812+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:44.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:44.120+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:44.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:44.846+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:45.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:45.120+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:45.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:45.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:46.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:46.099+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:46.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:46.896+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:47.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:47.059+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:47.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:47.899+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:48.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:48.069+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:48.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:48.855+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:49.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:49.099+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:49.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:49.841+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:50.095 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:50.095+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:50.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:50.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:51.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:51.062+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:51.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:51.832+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:52.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:52.049+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:52.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:52.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:53.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:53.089+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:53.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:53.814+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:54.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:54.105+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:54.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:54.828+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:55.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:55.067+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:55.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:55.870+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:56.113 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:56.112+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:56.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:56.908+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:57.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:57.076+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:57.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:57.868+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:58.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:58.064+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:58.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:58.869+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:59.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T18:59:59.064+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T18:59:59.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T18:59:59.871+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:00.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:00.021+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:00.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:00.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:01.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:01.058+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:01.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:01.889+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:02.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:02.051+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:02.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:02.935+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:03.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:03.030+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:03.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:03.946+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:04.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:04.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:04.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:04.981+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:05.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:05.025+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:06.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:06.003+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:06.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:06.076+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:06.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:06.973+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:07.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:07.096+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:07.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:07.960+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:08.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:08.077+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:08.218 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T19:00:08.218+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffb6400a020 msgr2=0x7ffb6c03d1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T19:00:08.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:08.975+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:09.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:09.049+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:10.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:10.019+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:10.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:10.081+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:11.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:11.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:11.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:11.083+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:12.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:12.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:12.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:12.101+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:13.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:13.078+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:13.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:13.090+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:14.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:14.042+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:14.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:14.111+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:15.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:15.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:15.148 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:15.148+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:15.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:15.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:16.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:16.108+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:16.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:16.920+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:17.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:17.087+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:17.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:17.944+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:18.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:18.074+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:18.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:18.912+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:19.085 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:19.085+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:19.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:19.935+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:20.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:20.136+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:20.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:20.952+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:21.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:21.129+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:21.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:21.950+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:22.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:22.085+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:22.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:22.986+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:23.061 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:23.060+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:23.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:23.970+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:24.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:24.075+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:25.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:25.010+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:25.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:25.119+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:26.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:26.041+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:26.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:26.143+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:27.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:27.038+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:27.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:27.101+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:28.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:28.003+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:28.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:28.126+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:29.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:29.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:29.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:29.153+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:30.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:30.026+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:30.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:30.200+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:31.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:31.073+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:31.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:31.232+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:32.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:32.099+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:32.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:32.252+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:33.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:33.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:33.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:33.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:34.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:34.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:34.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:34.258+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:35.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:35.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:35.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:35.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:36.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:36.053+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:36.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:36.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:37.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:37.097+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:37.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:37.295+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:38.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:38.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:38.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:38.292+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:39.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:39.065+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:39.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:39.273+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:40.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:40.080+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:40.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:40.286+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:41.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:41.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:41.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:41.273+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:42.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:42.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:42.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:42.292+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:43.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:43.054+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:43.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:43.302+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:44.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:44.058+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:44.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:44.261+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:45.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:45.074+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:45.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:45.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:46.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:46.083+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:46.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:46.209+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:47.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:47.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:47.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:47.229+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:48.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:48.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:48.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:48.257+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:49.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:49.160+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:49.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:49.238+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:50.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:50.207+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:50.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:50.215+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:51.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:51.192+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:51.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:51.257+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:52.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:52.149+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:52.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:52.229+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:53.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:53.185+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:53.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:53.236+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:54.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:54.198+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:54.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:54.199+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:55.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:55.194+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:55.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:55.212+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:56.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:56.172+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:56.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:56.230+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:57.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:57.168+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:57.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:57.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:58.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:58.162+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:58.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:58.226+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:59.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:00:59.182+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:00:59.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:00:59.187+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:00.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:00.198+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:00.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:00.198+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:01.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:01.166+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:01.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:01.174+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:02.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:02.156+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:02.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:02.174+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:03.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:03.157+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:03.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:03.204+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:04.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:04.132+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:04.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:04.175+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:05.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:05.146+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:05.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:05.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:06.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:06.111+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:06.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:06.139+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:07.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:07.092+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:07.121 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:07.121+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:08.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:08.069+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:08.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:08.094+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:09.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:09.070+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:09.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:09.139+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:10.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:10.022+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:10.123 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:10.124+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:11.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:11.031+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:11.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:11.172+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:12.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:12.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:12.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:12.213+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:12.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:12.973+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:13.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:13.171+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:13.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:13.985+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:14.123 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:14.123+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:15.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:15.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:15.142 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:15.143+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:16.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:16.022+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:16.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:16.171+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:17.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:17.030+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:17.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:17.170+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:18.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:18.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:18.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:18.207+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:18.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:18.970+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:19.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:19.190+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:19.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:19.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:20.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:20.204+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:21.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:21.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:21.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:21.218+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:21.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:21.974+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:22.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:22.175+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:22.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:22.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:23.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:23.180+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:23.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:23.986+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:24.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:24.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:24.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:24.955+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:25.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:25.222+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:25.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:25.933+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:26.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:26.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:26.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:26.980+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:27.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:27.284+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:28.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:28.013+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:28.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:28.267+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:29.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:29.037+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:29.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:29.226+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:30.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:30.038+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:30.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:30.250+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:31.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:31.083+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:31.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:31.259+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:32.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:32.122+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:32.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:32.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:33.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:33.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:33.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:33.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:34.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:34.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:34.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:34.264+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:35.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:35.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:35.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:35.239+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:36.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:36.113+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:36.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:36.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:37.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:37.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:37.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:37.297+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:38.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:38.143+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:38.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:38.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:39.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:39.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:39.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:39.220+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:40.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:40.145+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:40.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:40.198+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:41.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:41.105+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:41.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:41.209+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:42.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:42.089+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:42.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:42.215+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:43.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:43.113+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:43.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:43.234+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:44.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:44.121+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:44.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:44.254+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:45.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:45.096+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:45.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:45.248+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:46.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:46.055+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:46.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:46.218+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:47.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:47.017+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:47.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:47.189+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:48.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:48.031+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:48.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:48.161+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:49.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:49.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:49.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:49.184+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:49.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:49.993+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:50.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:50.190+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:51.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:51.014+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:51.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:51.204+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:52.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:52.042+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:52.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:52.157+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:53.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:53.031+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:53.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:53.201+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:54.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:54.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:54.164 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:54.163+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:55.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:55.040+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:55.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:55.153+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:56.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:56.088+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:56.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:56.151+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:57.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:57.138+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:57.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:57.161+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:58.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:58.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:58.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:58.190+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:59.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:01:59.194+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:01:59.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:01:59.240+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:00.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:00.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:00.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:00.239+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:01.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:01.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:01.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:01.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:02.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:02.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:02.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:02.247+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:03.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:03.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:03.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:03.235+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:04.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:04.249+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:04.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:04.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:05.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:05.233+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:05.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:05.259+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:06.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:06.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:06.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:06.257+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:07.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:07.174+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:07.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:07.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:08.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:08.210+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:08.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:08.253+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:09.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:09.166+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:09.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:09.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:10.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:10.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:10.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:10.322+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:11.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:11.181+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:11.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:11.341+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:12.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:12.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:12.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:12.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:13.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:13.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:13.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:13.359+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:14.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:14.125+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:14.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:14.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:15.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:15.090+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:15.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:15.279+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:16.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:16.128+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:16.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:16.264+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:17.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:17.122+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:17.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:17.313+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:18.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:18.139+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:18.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:18.353+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:19.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:19.093+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:19.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:19.364+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:20.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:20.105+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:20.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:20.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:21.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:21.101+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:21.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:21.342+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:22.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:22.078+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:22.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:22.341+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:23.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:23.099+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:23.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:23.380+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:24.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:24.075+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:24.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:24.384+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:25.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:25.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:25.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:25.363+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:26.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:26.084+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:26.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:26.333+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:27.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:27.075+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:27.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:27.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:28.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:28.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:28.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:28.305+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:29.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:29.070+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:29.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:29.294+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:30.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:30.113+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:30.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:30.298+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:31.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:31.153+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:31.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:31.295+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:32.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:32.169+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:32.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:32.309+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:33.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:33.163+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:33.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:33.312+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:34.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:34.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:34.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:34.274+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:35.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:35.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:35.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:35.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:36.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:36.159+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:36.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:36.321+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:37.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:37.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:37.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:37.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:38.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:38.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:38.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:38.362+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:39.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:39.131+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:39.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:39.347+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:40.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:40.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:40.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:40.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:41.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:41.181+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:41.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:41.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:42.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:42.184+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:42.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:42.288+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:43.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:43.232+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:43.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:43.303+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:44.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:44.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:44.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:44.322+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:45.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:45.197+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:45.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:45.348+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:46.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:46.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:46.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:46.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:47.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:47.157+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:47.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:47.386+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:48.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:48.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:48.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:48.370+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:49.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:49.214+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:49.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:49.351+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:50.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:50.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:50.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:50.343+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:51.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:51.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:51.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:51.302+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:52.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:52.180+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:52.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:52.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:53.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:53.175+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:53.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:53.346+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:54.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:54.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:54.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:54.346+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:55.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:55.269+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:55.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:55.340+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:56.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:56.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:56.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:56.344+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:57.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:57.308+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:57.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:57.366+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:58.310 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:58.310+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:58.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:58.370+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:59.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:02:59.261+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:02:59.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:02:59.378+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:00.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:00.297+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:00.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:00.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:01.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:01.318+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:01.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:01.360+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:02.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:02.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:02.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:02.352+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:03.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:03.246+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:03.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:03.324+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:04.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:04.230+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:04.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:04.277+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:05.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:05.220+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:05.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:05.327+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:06.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:06.238+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:06.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:06.282+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:07.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:07.248+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:07.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:07.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:08.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:08.269+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:08.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:08.269+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:09.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:09.255+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:09.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:09.310+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:10.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:10.286+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:10.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:10.316+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:11.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:11.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:11.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:11.292+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:12.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:12.270+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:12.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:12.342+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:13.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:13.297+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:13.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:13.354+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:14.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:14.250+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:14.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:14.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:15.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:15.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:15.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:15.425+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:16.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:16.198+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:16.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:16.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:17.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:17.172+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:17.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:17.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:18.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:18.160+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:18.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:18.545+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:19.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:19.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:19.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:19.543+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:20.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:20.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:20.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:20.581+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:21.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:21.168+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:21.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:21.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:22.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:22.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:22.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:22.628+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:23.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:23.226+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:23.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:23.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:24.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:24.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:24.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:24.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:25.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:25.284+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:25.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:25.664+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:26.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:26.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:26.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:26.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:27.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:27.339+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:27.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:27.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:28.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:28.313+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:28.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:28.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:29.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:29.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:29.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:29.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:30.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:30.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:30.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:30.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:31.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:31.239+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:31.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:31.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:32.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:32.225+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:32.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:32.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:33.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:33.196+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:33.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:33.601+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:34.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:34.201+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:34.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:34.639+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:35.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:35.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:35.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:35.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:36.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:36.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:36.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:36.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:37.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:37.217+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:37.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:37.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:38.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:38.239+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:38.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:38.742+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:39.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:39.194+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:39.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:39.751+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:40.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:40.172+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:40.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:40.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:41.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:41.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:41.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:41.780+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:42.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:42.183+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:42.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:42.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:43.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:43.174+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:43.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:43.747+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:44.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:44.222+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:44.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:44.758+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:45.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:45.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:45.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:45.788+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:46.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:46.226+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:46.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:46.813+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:47.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:47.189+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:47.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:47.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:48.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:48.182+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:48.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:48.841+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:49.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:49.226+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:49.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:49.802+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:50.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:50.186+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:50.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:50.785+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:51.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:51.153+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:51.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:51.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:52.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:52.106+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:52.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:52.746+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:53.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:53.124+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:53.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:53.700+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:54.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:54.169+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:54.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:54.651+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:55.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:55.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:55.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:55.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:56.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:56.179+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:56.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:56.645+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:57.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:57.203+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:57.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:57.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:58.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:58.249+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:58.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:58.619+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:59.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:03:59.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:03:59.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:03:59.609+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:00.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:00.235+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:00.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:00.623+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:01.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:01.256+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:01.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:01.574+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:02.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:02.291+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:02.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:02.603+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:03.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:03.252+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:03.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:03.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:04.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:04.284+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:04.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:04.594+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:05.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:05.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:05.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:05.547+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:06.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:06.318+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:06.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:06.521+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:07.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:07.359+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:07.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:07.563+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:08.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:08.333+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:08.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:08.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:09.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:09.367+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:09.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:09.653+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:10.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:10.370+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:10.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:10.656+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:11.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:11.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:11.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:11.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:12.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:12.369+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:12.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:12.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:13.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:13.329+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:13.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:13.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:14.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:14.366+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:14.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:14.585+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:15.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:15.322+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:15.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:15.572+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:16.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:16.325+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:16.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:16.553+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:17.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:17.346+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:17.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:17.555+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:18.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:18.351+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:18.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:18.598+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:19.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:19.397+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:19.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:19.639+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:20.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:20.348+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:20.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:20.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:21.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:21.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:21.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:21.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:22.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:22.333+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:22.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:22.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:23.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:23.297+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:23.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:23.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:24.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:24.339+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:24.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:24.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:25.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:25.382+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:25.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:25.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:26.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:26.358+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:26.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:26.720+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:27.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:27.329+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:27.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:27.757+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:28.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:28.352+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:28.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:28.769+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:29.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:29.342+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:29.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:29.779+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:30.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:30.318+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:30.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:30.789+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:31.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:31.311+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:31.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:31.798+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:32.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:32.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:32.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:32.760+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:33.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:33.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:33.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:33.786+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:34.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:34.306+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:34.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:34.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:35.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:35.357+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:35.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:35.786+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:36.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:36.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:36.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:36.830+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:37.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:37.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:37.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:37.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:38.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:38.338+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:38.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:38.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:39.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:39.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:39.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:39.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:40.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:40.295+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:40.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:40.815+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:41.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:41.287+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:41.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:41.819+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:42.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:42.320+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:42.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:42.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:43.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:43.295+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:43.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:43.906+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:44.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:44.259+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:44.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:44.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:45.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:45.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:45.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:45.902+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:46.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:46.280+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:46.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:46.939+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:47.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:47.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:47.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:47.976+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:48.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:48.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:48.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:48.989+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:49.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:49.283+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:49.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:49.958+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:50.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:50.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:51.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:51.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:51.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:51.268+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:51.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:51.973+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:52.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:52.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:52.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:52.970+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:53.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:53.201+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:53.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:53.926+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:54.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:54.160+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:54.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:54.966+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:55.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:55.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:55.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:55.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:56.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:56.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:57.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:57.007+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:57.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:57.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:57.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:57.989+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:58.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:58.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:59.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:04:59.024+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:04:59.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:04:59.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:00.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:00.054+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:00.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:00.230+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:01.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:01.091+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:01.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:01.264+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:02.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:02.066+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:02.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:02.285+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:03.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:03.034+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:03.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:03.255+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:04.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:04.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:04.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:04.258+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:05.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:05.086+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:05.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:05.233+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:06.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:06.071+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:06.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:06.211+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:07.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:07.116+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:07.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:07.241+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:08.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:08.110+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:08.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:08.220+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:09.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:09.105+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:09.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:09.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:10.142 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:10.142+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:10.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:10.282+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:11.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:11.170+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:11.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:11.280+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:12.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:12.154+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:12.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:12.284+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:13.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:13.143+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:13.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:13.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:14.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:14.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:14.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:14.302+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:15.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:15.181+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:15.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:15.311+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:16.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:16.211+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:16.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:16.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:17.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:17.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:17.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:17.287+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:18.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:18.223+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:18.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:18.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:19.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:19.223+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:19.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:19.319+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:20.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:20.274+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:20.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:20.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:21.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:21.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:21.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:21.338+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:22.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:22.303+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:22.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:22.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:23.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:23.350+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:23.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:23.368+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:24.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:24.350+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:24.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:24.397+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:25.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:25.380+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:25.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:25.438+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:26.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:26.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:26.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:26.434+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:27.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:27.353+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:27.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:27.455+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:28.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:28.309+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:28.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:28.413+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:29.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:29.316+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:29.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:29.398+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:30.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:30.350+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:30.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:30.439+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:31.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:31.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:31.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:31.425+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:32.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:32.392+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:32.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:32.432+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:33.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:33.417+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:33.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:33.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:34.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:34.414+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:34.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:34.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:35.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:35.440+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:35.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:35.490+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:36.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:36.446+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:36.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:36.529+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:37.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:37.476+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:37.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:37.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:38.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:38.460+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:38.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:38.470+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:39.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:39.415+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:39.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:39.499+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:40.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:40.412+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:40.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:40.466+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:41.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:41.438+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:41.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:41.511+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:42.457 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:42.457+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:42.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:42.468+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:43.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:43.444+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:43.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:43.476+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:44.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:44.431+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:44.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:44.452+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:45.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:45.390+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:45.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:45.486+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:46.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:46.411+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:46.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:46.490+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:47.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:47.390+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:47.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:47.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:48.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:48.368+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:48.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:48.477+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:49.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:49.330+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:49.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:49.473+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:50.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:50.314+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:50.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:50.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:51.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:51.347+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:51.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:51.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:52.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:52.381+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:52.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:52.558+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:53.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:53.380+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:53.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:53.577+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:54.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:54.403+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:54.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:54.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:55.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:55.422+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:55.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:55.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:56.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:56.423+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:56.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:56.609+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:57.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:57.432+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:57.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:57.658+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:58.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:58.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:58.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:58.610+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:59.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:05:59.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:05:59.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:05:59.600+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:00.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:00.347+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:00.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:00.593+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:01.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:01.354+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:01.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:01.600+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:02.403 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:02.403+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:02.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:02.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:03.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:03.441+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:03.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:03.647+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:04.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:04.468+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:04.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:04.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:05.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:05.435+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:05.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:05.617+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:06.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:06.433+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:06.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:06.647+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:07.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:07.420+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:07.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:07.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:08.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:08.431+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:08.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:08.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:09.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:09.382+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:09.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:09.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:10.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:10.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:10.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:10.704+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:11.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:11.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:11.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:11.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:12.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:12.354+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:12.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:12.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:13.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:13.325+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:13.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:13.732+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:14.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:14.329+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:14.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:14.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:15.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:15.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:15.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:15.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:16.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:16.276+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:16.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:16.734+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:17.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:17.272+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:17.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:17.764+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:18.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:18.256+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:18.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:18.761+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:19.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:19.237+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:19.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:19.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:20.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:20.241+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:20.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:20.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:21.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:21.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:21.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:21.675+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:22.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:22.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:22.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:22.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:23.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:23.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:23.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:23.740+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:24.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:24.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:24.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:24.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:25.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:25.295+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:25.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:25.733+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:26.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:26.326+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:26.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:26.740+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:27.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:27.340+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:27.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:27.740+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:28.378 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:28.378+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:28.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:28.781+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:29.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:29.343+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:29.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:29.759+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:30.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:30.333+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:30.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:30.802+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:31.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:31.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:31.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:31.784+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:32.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:32.252+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:32.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:32.819+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:33.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:33.203+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:33.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:33.804+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:34.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:34.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:34.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:34.826+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:35.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:35.201+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:35.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:35.845+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:36.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:36.217+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:36.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:36.817+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:37.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:37.242+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:37.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:37.807+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:38.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:38.286+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:38.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:38.820+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:39.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:39.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:39.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:39.800+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:40.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:40.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:40.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:40.837+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:41.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:41.249+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:41.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:41.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:42.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:42.274+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:42.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:42.807+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:43.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:43.249+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:43.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:43.825+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:44.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:44.262+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:44.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:44.854+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:45.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:45.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:45.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:45.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:46.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:46.289+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:46.924 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:46.924+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:47.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:47.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:47.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:47.974+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:48.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:48.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:48.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:48.926+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:49.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:49.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:49.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:49.920+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:50.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:50.231+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:50.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:50.948+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:51.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:51.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:51.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:51.973+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:52.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:52.268+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:53.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:53.002+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:53.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:53.262+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:54.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:54.023+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:54.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:54.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:55.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:55.063+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:55.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:55.256+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:56.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:56.074+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:56.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:56.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:57.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:57.063+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:57.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:57.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:58.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:58.016+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:58.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:58.299+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:59.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:06:59.027+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:06:59.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:06:59.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:00.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:00.069+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:00.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:00.288+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:01.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:01.095+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:01.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:01.249+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:02.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:02.072+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:02.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:02.222+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:03.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:03.037+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:03.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:03.179+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:03.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:03.990+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:04.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:04.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:05.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:05.008+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:05.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:05.130+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:05.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:05.981+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:06.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:06.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:06.935 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:06.936+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:07.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:07.115+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:07.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:07.984+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:08.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:08.147+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:08.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:08.949+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:09.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:09.119+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:09.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:09.979+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:10.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:10.099+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:10.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:10.991+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:11.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:11.065+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:11.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:11.984+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:12.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:12.016+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:12.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:12.952+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:13.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:13.031+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:13.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:13.986+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:14.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:14.020+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:14.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:14.993+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:15.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:15.055+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:16.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:16.030+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:16.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:16.075+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:17.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:17.008+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:17.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:17.029+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:18.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:18.053+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:18.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:18.058+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:19.047 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:19.047+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:19.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:19.053+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:20.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:20.028+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:20.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:20.060+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:20.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:20.997+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:21.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:21.083+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:21.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:21.977+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:22.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:22.108+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:22.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:22.975+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:23.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:23.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:24.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:24.022+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:24.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:24.109+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:25.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:25.050+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:25.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:25.095+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:26.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:26.051+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:26.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:26.084+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:27.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:27.030+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:27.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:27.046+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:28.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:28.034+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:28.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:28.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:29.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:29.000+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:29.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:29.022+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:29.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:29.988+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:30.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:30.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:30.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:30.942+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:31.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:31.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:31.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:31.925+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:32.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:32.011+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:32.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:32.972+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:32.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:32.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:33.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:33.957+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:33.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:33.995+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:34.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:34.927+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:34.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:34.981+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:35.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:35.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:35.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:35.955+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:36.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:36.915+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:36.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:36.941+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:37.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:37.876+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:37.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:37.900+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:38.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:38.852+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:38.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:38.867+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:39.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:39.829+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:39.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:39.845+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:40.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:40.845+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:40.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:40.857+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:41.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:41.844+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:41.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:41.892+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:42.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:42.893+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:42.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:42.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:43.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:43.920+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:43.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:43.921+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:44.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:44.873+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:44.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:44.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:45.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:45.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:45.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:45.955+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:46.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:46.902+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:46.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:46.984+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:47.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:47.888+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:48.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:48.003+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:48.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:48.888+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:49.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:49.013+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:49.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:49.899+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:49.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:49.987+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:50.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:50.926+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:51.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:51.026+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:51.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:51.971+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:51.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:51.990+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:53.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:53.003+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:53.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:53.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:54.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:54.027+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:54.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:54.046+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:55.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:55.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:55.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:55.045+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:56.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:56.020+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:56.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:56.029+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:57.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:57.012+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:57.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:57.036+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:58.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:58.036+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:58.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:58.062+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:59.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:07:59.012+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:07:59.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:07:59.030+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:00.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:00.005+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:00.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:00.039+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:01.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:01.047+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:01.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:01.072+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:02.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:02.005+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:02.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:02.046+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:02.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:02.963+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:03.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:03.009+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:03.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:03.926+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:03.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:03.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:04.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:04.883+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:04.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:04.972+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:05.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:05.838+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:05.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:05.994+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:06.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:06.818+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:07.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:06.951+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:07.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:07.849+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:07.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:07.973+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:08.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:08.826+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:08.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:08.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:09.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:09.834+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:09.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:09.941+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:10.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:10.874+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:10.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:10.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:11.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:11.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:11.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:11.937+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:12.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:12.901+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:12.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:12.911+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:13.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:13.875+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:13.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:13.951+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:14.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:14.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:14.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:14.931+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:15.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:15.902+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:15.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:15.963+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:16.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:16.936+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:16.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:16.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:17.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:17.940+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:17.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:17.946+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:18.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:18.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:18.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:18.945+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:19.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:19.914+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:19.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:19.933+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:20.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:20.894+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:20.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:20.934+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:21.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:21.890+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:21.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:21.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:22.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:22.858+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:22.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:22.906+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:23.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:23.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:23.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:23.908+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:24.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:24.817+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:24.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:24.865+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:25.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:25.830+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:25.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:25.842+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:26.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:26.788+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:26.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:26.866+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:27.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:27.791+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:27.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:27.820+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:28.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:28.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:28.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:28.799+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:29.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:29.752+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:29.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:29.782+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:30.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:30.734+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:30.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:30.823+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:31.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:31.709+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:31.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:31.850+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:32.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:32.742+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:32.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:32.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:33.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:33.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:33.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:33.861+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:34.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:34.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:34.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:34.848+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:35.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:35.741+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:35.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:35.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:36.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:36.774+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:36.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:36.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:37.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:37.750+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:37.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:37.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:38.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:38.787+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:38.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:38.869+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:39.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:39.767+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:39.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:39.876+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:40.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:40.785+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:40.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:40.921+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:41.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:41.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:41.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:41.920+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:42.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:42.816+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:42.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:42.952+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:43.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:43.768+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:43.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:43.944+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:44.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:44.731+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:44.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:44.976+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:45.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:45.722+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:46.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:46.014+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:46.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:46.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:47.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:47.014+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:47.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:47.744+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:48.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:48.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:48.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:48.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:49.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:49.028+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:49.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:49.744+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:49.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:49.993+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:50.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:50.754+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:50.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:50.957+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:51.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:51.763+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:51.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:51.990+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:52.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:52.790+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:52.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:52.979+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:53.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:53.751+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:54.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:54.014+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:54.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:54.716+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:54.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:54.989+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:55.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:55.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:55.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:55.982+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:56.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:56.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:56.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:56.937+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:57.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:57.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:57.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:57.909+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:58.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:58.660+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:58.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:58.948+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:59.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:08:59.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:08:59.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:08:59.969+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:00.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:00.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:00.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:00.938+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:01.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:01.658+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:01.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:01.983+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:02.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:02.610+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:02.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:02.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:03.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:03.626+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:03.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:03.958+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:04.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:04.629+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:04.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:04.985+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:05.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:05.592+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:05.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:06.000+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:06.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:06.631+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:06.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:06.993+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:07.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:07.629+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:07.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:07.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:08.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:08.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:08.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:08.936+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:09.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:09.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:09.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:09.915+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:10.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:10.672+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:10.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:10.940+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:11.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:11.709+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:11.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:11.967+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:12.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:12.711+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:13.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:13.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:13.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:13.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:13.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:13.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:14.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:14.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:14.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:14.939+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:15.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:15.711+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:15.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:15.965+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:16.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:16.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:16.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:16.935+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:17.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:17.702+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:17.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:17.938+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:18.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:18.702+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:18.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:18.952+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:19.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:19.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:19.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:19.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:20.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:20.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:20.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:20.967+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:21.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:21.655+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:21.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:21.918+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:22.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:22.702+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:22.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:22.884+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:23.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:23.704+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:23.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:23.929+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:24.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:24.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:24.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:24.915+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:25.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:25.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:25.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:25.904+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:26.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:26.662+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:26.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:26.914+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:27.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:27.708+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:27.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:27.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:28.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:28.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:29.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:29.001+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:29.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:29.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:30.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:30.043+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:30.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:30.790+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:31.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:31.025+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:31.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:31.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:32.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:32.015+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:32.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:32.801+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:33.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:33.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:33.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:33.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:33.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:33.973+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:34.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:34.829+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:35.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:34.953+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:35.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:35.787+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:35.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:35.932+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:36.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:36.822+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:36.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:36.921+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:37.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:37.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:37.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:37.885+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:38.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:38.758+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:38.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:38.857+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:39.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:39.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:39.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:39.896+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:40.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:40.670+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:40.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:40.863+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:41.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:41.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:41.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:41.863+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:42.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:42.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:42.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:42.827+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:43.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:43.715+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:43.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:43.841+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:44.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:44.750+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:44.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:44.859+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:45.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:45.758+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:45.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:45.829+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:46.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:46.744+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:46.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:46.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:47.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:47.764+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:47.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:47.844+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:48.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:48.813+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:48.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:48.880+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:49.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:49.770+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:49.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:49.849+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:50.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:50.755+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:50.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:50.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:51.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:51.790+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:51.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:51.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:52.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:52.798+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:52.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:52.821+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:53.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:53.767+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:53.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:53.867+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:54.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:54.768+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:54.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:54.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:55.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:55.809+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:55.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:55.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:56.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:56.784+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:56.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:56.946+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:57.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:57.751+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:57.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:57.970+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:58.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:58.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:58.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:09:58.995+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:09:59.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:09:59.808+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:00.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:00.039+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:00.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:00.856+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:01.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:01.004+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:01.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:01.903+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:01.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:01.960+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:02.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:02.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:02.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:02.930+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:03.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:03.879+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:03.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:03.916+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:04.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:04.887+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:04.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:04.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:05.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:05.910+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:05.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:05.937+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:06.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:06.923+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:06.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:06.947+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:07.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:07.945+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:07.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:07.971+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:08.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:08.929+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:08.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:08.940+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:09.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:09.919+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:09.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:09.947+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:10.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:10.939+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:10.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:10.970+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:11.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:11.922+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:11.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:11.961+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:12.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:12.917+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:12.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:12.968+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:13.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:13.912+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:13.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:13.931+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:14.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:14.889+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:14.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:14.947+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:15.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:15.874+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:15.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:15.899+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:16.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:16.889+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:16.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:16.946+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:17.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:17.886+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:17.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:17.979+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:18.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:18.863+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:18.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:18.948+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:19.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:19.907+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:19.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:19.969+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:20.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:20.919+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:20.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:20.955+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:21.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:21.915+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:21.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:21.952+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:22.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:22.944+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:22.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:22.955+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:23.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:23.984+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:23.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:23.992+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:24.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:24.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:24.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:24.996+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:26.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:26.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:26.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:26.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:26.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:26.967+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:26.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:26.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:27.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:27.948+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:28.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:28.025+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:28.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:28.933+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:29.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:29.013+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:29.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:29.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:29.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:29.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:30.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:30.967+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:31.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:31.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:31.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:31.920+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:32.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:32.014+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:32.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:32.888+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:32.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:32.980+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:33.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:33.856+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:34.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:34.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:34.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:34.873+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:35.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:35.059+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:35.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:35.844+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:36.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:36.048+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:36.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:36.871+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:37.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:37.026+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:37.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:37.919+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:38.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:38.041+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:38.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:38.921+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:39.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:39.057+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:39.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:39.874+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:40.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:40.068+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:40.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:40.836+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:41.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:41.116+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:41.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:41.859+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:42.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:42.147+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:42.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:42.877+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:43.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:43.192+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:43.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:43.892+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:44.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:44.206+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:44.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:44.926+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:45.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:45.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:45.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:45.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:46.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:46.247+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:46.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:46.943+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:47.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:47.273+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:47.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:47.969+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:48.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:48.237+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:48.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:48.997+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:49.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:49.235+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:50.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:50.028+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:50.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:50.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:51.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:51.063+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:51.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:51.213+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:52.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:52.037+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:52.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:52.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:53.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:53.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:53.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:53.235+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:54.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:54.024+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:54.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:54.260+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:55.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:55.053+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:55.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:55.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:56.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:56.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:56.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:56.251+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:57.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:57.039+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:57.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:57.217+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:58.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:58.032+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:58.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:58.181+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:59.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:10:59.035+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:10:59.183 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:10:59.184+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:00.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:00.051+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:00.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:00.182+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:01.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:01.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:01.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:01.231+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:02.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:02.098+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:02.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:02.194+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:03.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:03.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:03.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:03.154+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:04.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:04.062+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:04.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:04.182+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:05.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:05.064+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:05.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:05.160+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:06.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:06.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:06.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:06.158+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:07.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:07.125+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:07.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:07.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:08.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:08.119+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:08.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:08.150+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:09.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:09.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:09.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:09.156+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:10.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:10.116+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:10.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:10.161+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:11.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:11.081+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:11.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:11.119+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:12.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:12.080+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:12.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:12.123+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:13.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:13.079+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:13.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:13.114+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:14.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:14.067+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:14.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:14.120+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:15.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:15.096+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:15.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:15.157+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:16.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:16.138+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:16.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:16.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:17.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:17.107+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:17.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:17.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:18.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:18.103+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:18.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:18.134+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:19.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:19.073+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:19.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:19.129+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:20.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:20.072+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:20.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:20.166+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:21.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:21.064+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:21.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:21.208+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:22.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:22.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:22.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:22.248+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:23.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:23.118+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:23.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:23.236+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:24.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:24.168+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:24.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:24.249+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:25.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:25.138+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:25.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:25.230+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:26.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:26.185+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:26.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:26.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:27.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:27.232+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:27.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:27.266+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:28.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:28.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:28.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:28.304+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:29.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:29.300+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:29.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:29.352+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:30.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:30.254+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:30.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:30.349+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:31.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:31.304+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:31.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:31.399+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:32.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:32.304+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:32.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:32.423+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:33.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:33.289+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:33.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:33.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:34.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:34.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:34.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:34.477+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:35.317 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:35.317+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:35.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:35.507+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:36.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:36.314+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:36.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:36.531+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:37.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:37.295+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:37.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:37.483+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:38.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:38.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:38.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:38.471+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:39.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:39.232+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:39.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:39.489+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:40.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:40.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:40.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:40.512+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:41.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:41.283+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:41.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:41.477+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:42.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:42.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:42.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:42.438+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:43.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:43.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:43.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:43.415+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:44.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:44.221+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:44.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:44.445+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:45.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:45.230+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:45.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:45.420+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:46.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:46.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:46.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:46.424+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:47.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:47.201+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:47.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:47.377+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:48.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:48.220+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:48.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:48.345+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:49.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:49.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:49.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:49.329+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:50.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:50.211+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:50.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:50.325+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:51.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:51.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:51.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:51.345+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:52.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:52.236+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:52.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:52.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:53.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:53.277+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:53.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:53.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:54.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:54.234+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:54.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:54.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:55.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:55.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:55.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:55.307+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:56.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:56.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:56.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:56.317+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:57.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:57.161+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:57.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:57.299+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:58.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:58.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:58.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:58.258+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:59.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:11:59.169+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:11:59.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:11:59.212+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:00.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:00.164+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:00.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:00.207+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:01.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:01.161+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:01.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:01.200+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:02.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:02.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:02.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:02.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:03.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:03.185+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:03.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:03.276+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:04.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:04.169+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:04.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:04.294+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:05.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:05.187+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:05.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:05.337+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:06.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:06.160+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:06.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:06.343+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:07.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:07.191+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:07.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:07.381+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:08.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:08.174+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:08.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:08.379+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:09.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:09.172+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:09.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:09.424+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:10.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:10.206+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:10.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:10.376+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:11.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:11.209+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:11.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:11.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:12.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:12.200+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:12.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:12.431+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:13.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:13.224+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:13.427 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:13.427+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:14.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:14.231+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:14.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:14.406+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:15.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:15.235+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:15.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:15.429+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:16.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:16.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:16.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:16.409+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:17.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:17.318+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:17.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:17.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:18.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:18.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:18.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:18.364+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:19.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:19.338+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:19.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:19.360+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:20.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:20.326+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:20.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:20.348+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:21.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:21.329+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:21.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:21.348+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:22.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:22.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:22.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:22.322+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:23.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:23.283+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:23.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:23.346+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:24.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:24.308+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:24.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:24.316+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:25.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:25.297+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:25.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:25.300+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:26.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:26.255+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:26.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:26.289+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:27.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:27.298+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:27.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:27.339+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:28.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:28.325+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:28.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:28.335+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:29.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:29.287+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:29.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:29.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:30.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:30.245+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:30.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:30.289+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:31.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:31.231+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:31.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:31.290+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:32.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:32.218+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:32.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:32.281+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:33.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:33.195+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:33.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:33.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:34.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:34.196+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:34.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:34.218+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:35.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:35.187+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:35.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:35.197+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:36.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:36.211+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:36.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:36.214+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:37.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:37.197+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:37.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:37.217+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:38.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:38.189+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:38.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:38.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:39.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:39.216+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:39.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:39.234+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:40.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:40.210+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:40.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:40.220+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:41.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:41.233+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:41.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:41.240+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:42.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:42.203+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:42.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:42.267+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:43.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:43.208+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:43.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:43.271+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:44.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:44.251+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:44.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:44.301+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:45.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:45.266+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:45.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:45.320+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:46.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:46.231+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:46.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:46.352+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:47.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:47.273+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:47.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:47.345+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:48.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:48.259+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:48.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:48.392+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:49.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:49.277+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:49.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:49.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:50.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:50.277+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:50.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:50.334+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:51.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:51.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:51.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:51.297+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:52.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:52.264+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:52.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:52.300+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:53.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:53.270+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:53.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:53.277+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:54.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:54.278+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:54.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:54.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:55.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:55.255+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:55.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:55.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:56.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:56.264+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:56.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:56.303+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:57.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:57.283+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:57.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:57.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:58.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:58.280+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:58.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:58.288+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:59.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:12:59.264+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:12:59.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:12:59.285+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:00.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:00.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:00.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:00.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:01.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:01.250+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:01.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:01.312+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:02.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:02.205+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:02.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:02.296+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:03.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:03.208+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:03.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:03.340+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:04.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:04.233+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:04.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:04.365+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:05.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:05.202+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:05.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:05.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:06.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:06.179+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:06.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:06.409+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:07.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:07.133+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:07.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:07.453+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:08.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:08.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:08.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:08.484+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:09.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:09.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:09.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:09.465+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:10.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:10.153+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:10.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:10.474+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:11.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:11.189+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:11.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:11.497+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:12.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:12.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:12.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:12.515+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:13.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:13.155+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:13.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:13.528+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:14.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:14.193+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:14.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:14.574+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:15.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:15.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:15.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:15.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:16.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:16.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:16.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:16.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:17.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:17.100+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:17.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:17.634+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:18.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:18.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:18.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:18.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:19.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:19.079+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:19.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:19.628+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:20.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:20.042+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:20.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:20.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:20.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:20.997+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:21.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:21.618+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:22.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:22.027+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:22.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:22.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:22.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:22.978+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:23.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:23.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:23.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:23.959+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:24.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:24.655+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:24.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:24.940+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:25.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:25.622+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:25.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:25.988+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:26.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:26.664+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:27.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:27.026+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:27.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:27.621+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:28.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:28.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:28.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:28.607+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:29.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:29.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:29.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:29.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:29.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:29.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:30.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:30.600+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:30.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:30.966+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:31.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:31.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:31.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:31.949+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:32.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:32.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:32.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:32.984+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:33.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:33.580+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:33.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:33.948+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:34.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:34.584+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:35.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:34.999+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:35.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:35.587+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:36.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:36.032+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:36.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:36.611+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:37.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:37.053+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:37.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:37.653+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:38.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:38.056+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:38.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:38.610+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:39.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:39.038+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:39.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:39.598+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:40.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:40.087+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:40.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:40.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:41.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:41.130+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:41.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:41.607+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:42.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:42.090+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:42.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:42.596+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:43.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:43.140+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:43.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:43.549+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:44.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:44.165+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:44.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:44.513+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:45.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:45.183+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:45.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:45.477+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:46.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:46.199+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:46.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:46.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:47.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:47.210+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:47.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:47.521+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:48.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:48.244+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:48.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:48.533+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:49.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:49.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:49.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:49.536+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:50.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:50.220+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:50.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:50.576+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:51.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:51.245+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:51.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:51.617+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:52.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:52.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:52.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:52.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:53.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:53.180+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:53.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:53.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:54.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:54.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:54.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:54.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:55.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:55.220+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:55.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:55.684+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:56.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:56.198+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:56.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:56.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:57.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:57.210+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:57.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:57.653+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:58.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:58.181+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:58.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:58.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:59.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:13:59.143+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:13:59.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:13:59.610+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:00.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:00.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:00.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:00.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:01.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:01.180+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:01.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:01.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:02.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:02.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:02.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:02.678+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:03.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:03.158+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:03.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:03.675+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:04.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:04.135+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:04.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:04.718+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:05.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:05.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:05.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:05.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:06.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:06.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:06.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:06.721+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:07.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:07.176+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:07.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:07.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:08.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:08.212+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:08.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:08.680+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:09.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:09.252+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:09.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:09.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:10.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:10.211+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:10.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:10.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:11.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:11.191+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:11.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:11.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:12.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:12.189+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:12.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:12.704+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:13.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:13.218+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:13.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:13.721+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:14.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:14.238+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:14.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:14.700+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:15.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:15.209+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:15.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:15.715+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:16.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:16.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:16.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:16.719+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:17.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:17.207+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:17.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:17.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:18.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:18.207+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:18.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:18.734+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:19.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:19.229+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:19.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:19.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:20.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:20.187+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:20.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:20.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:21.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:21.171+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:21.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:21.718+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:22.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:22.206+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:22.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:22.748+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:23.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:23.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:23.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:23.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:24.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:24.175+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:24.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:24.769+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:25.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:25.178+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:25.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:25.820+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:26.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:26.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:26.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:26.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:27.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:27.170+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:27.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:27.810+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:28.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:28.146+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:28.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:28.829+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:29.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:29.133+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:29.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:29.833+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:30.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:30.102+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:30.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:30.859+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:31.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:31.127+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:31.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:31.834+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:32.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:32.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:32.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:32.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:33.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:33.104+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:33.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:33.831+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:34.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:34.109+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:34.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:34.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:35.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:35.144+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:35.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:35.901+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:36.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:36.190+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:36.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:36.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:37.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:37.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:37.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:37.919+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:38.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:38.268+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:38.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:38.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:39.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:39.225+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:39.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:39.891+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:40.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:40.191+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:40.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:40.891+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:41.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:41.189+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:41.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:41.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:42.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:42.224+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:42.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:42.867+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:43.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:43.216+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:43.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:43.906+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:44.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:44.183+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:44.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:44.866+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:45.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:45.232+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:45.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:45.887+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:46.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:46.188+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:46.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:46.867+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:47.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:47.150+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:47.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:47.863+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:48.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:48.114+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:48.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:48.858+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:49.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:49.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:49.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:49.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:50.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:50.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:50.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:50.859+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:51.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:51.074+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:51.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:51.883+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:52.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:52.069+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:52.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:52.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:53.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:53.119+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:53.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:53.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:54.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:54.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:54.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:54.868+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:55.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:55.081+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:55.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:55.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:56.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:56.080+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:56.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:56.880+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:57.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:57.057+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:57.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:57.897+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:58.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:58.020+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:58.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:58.896+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:58.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:58.997+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:59.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:14:59.889+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:14:59.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:14:59.997+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:00.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:00.891+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:01.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:01.021+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:01.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:01.912+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:01.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:01.979+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:02.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:02.922+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:03.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:03.002+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:03.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:03.924+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:04.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:04.039+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:04.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:04.890+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:05.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:05.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:05.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:05.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:06.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:06.005+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:06.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:06.905+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:06.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:06.986+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:07.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:07.937+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:08.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:08.019+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:08.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:08.917+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:08.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:08.978+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:09.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:09.909+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:09.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:09.949+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:10.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:10.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:10.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:10.950+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:11.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:11.903+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:11.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:11.991+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:12.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:12.881+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:12.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:12.985+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:13.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:13.835+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:13.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:13.944+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:14.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:14.872+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:14.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:14.915+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:15.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:15.831+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:15.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:15.904+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:16.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:16.831+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:16.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:16.868+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:17.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:17.788+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:17.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:17.871+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:18.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:18.751+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:18.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:18.882+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:19.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:19.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:19.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:19.915+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:20.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:20.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:20.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:20.922+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:21.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:21.697+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:21.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:21.942+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:22.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:22.661+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:22.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:22.914+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:23.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:23.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:23.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:23.958+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:24.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:24.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:25.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:25.007+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:25.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:25.687+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:26.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:26.050+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:26.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:26.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:27.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:27.021+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:27.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:27.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:28.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:28.008+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:28.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:28.683+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:28.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:28.976+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:29.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:29.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:29.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:29.996+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:30.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:30.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:31.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:31.023+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:31.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:31.629+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:32.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:32.018+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:32.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:32.644+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:32.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:32.978+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:33.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:33.658+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:33.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:33.961+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:34.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:34.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:34.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:34.960+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:35.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:35.660+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:35.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:35.942+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:36.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:36.628+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:36.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:36.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:37.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:37.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:37.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:37.918+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:38.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:38.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:38.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:38.954+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:39.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:39.610+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:39.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:39.981+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:40.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:40.567+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:40.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:40.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:41.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:41.567+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:41.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:41.971+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:42.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:42.578+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:42.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:42.982+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:43.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:43.533+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:43.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:43.990+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:44.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:44.488+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:44.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:44.948+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:45.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:45.510+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:45.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:45.971+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:46.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:46.544+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:46.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:46.977+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:47.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:47.568+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:47.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:47.947+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:48.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:48.590+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:48.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:48.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:49.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:49.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:50.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:50.004+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:50.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:50.513+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:50.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:50.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:51.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:51.525+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:51.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:51.989+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:52.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:52.575+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:53.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:53.003+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:53.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:53.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:53.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:53.981+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:54.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:54.633+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:55.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:55.030+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:55.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:55.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:56.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:56.067+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:56.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:56.586+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:57.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:57.048+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:57.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:57.546+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:58.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:58.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:58.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:58.588+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:59.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:15:59.129+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:15:59.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:15:59.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:00.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:00.134+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:00.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:00.586+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:01.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:01.154+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:01.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:01.547+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:02.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:02.150+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:02.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:02.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:03.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:03.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:03.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:03.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:04.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:04.126+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:04.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:04.545+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:05.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:05.172+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:05.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:05.535+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:06.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:06.137+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:06.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:06.544+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:07.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:07.102+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:07.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:07.515+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:08.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:08.087+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:08.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:08.532+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:09.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:09.123+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:09.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:09.542+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:10.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:10.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:10.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:10.554+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:11.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:11.122+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:11.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:11.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:12.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:12.113+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:12.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:12.623+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:13.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:13.069+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:13.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:13.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:14.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:14.046+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:14.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:14.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:15.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:15.041+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:15.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:15.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:16.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:16.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:16.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:16.626+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:17.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:17.092+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:17.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:17.603+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:18.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:18.119+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:18.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:18.581+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:19.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:19.101+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:19.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:19.624+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:20.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:20.122+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:20.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:20.619+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:21.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:21.167+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:21.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:21.626+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:22.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:22.195+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:22.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:22.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:23.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:23.175+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:23.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:23.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:24.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:24.204+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:24.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:24.691+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:25.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:25.162+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:25.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:25.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:26.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:26.181+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:26.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:26.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:27.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:27.173+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:27.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:27.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:28.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:28.221+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:28.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:28.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:29.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:29.253+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:29.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:29.562+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:30.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:30.222+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:30.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:30.602+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:31.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:31.227+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:31.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:31.606+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:32.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:32.219+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:32.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:32.593+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:33.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:33.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:33.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:33.567+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:34.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:34.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:34.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:34.549+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:35.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:35.303+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:35.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:35.563+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:36.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:36.278+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:36.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:36.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:37.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:37.258+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:37.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:37.583+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:38.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:38.223+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:38.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:38.626+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:39.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:39.247+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:39.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:39.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:40.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:40.285+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:40.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:40.660+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:41.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:41.279+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:41.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:41.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:42.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:42.314+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:42.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:42.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:43.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:43.268+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:43.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:43.759+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:44.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:44.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:44.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:44.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:45.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:45.265+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:45.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:45.761+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:46.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:46.282+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:46.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:46.805+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:47.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:47.262+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:47.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:47.828+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:48.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:48.312+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:48.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:48.878+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:49.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:49.342+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:49.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:49.883+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:50.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:50.321+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:50.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:50.917+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:51.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:51.291+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:51.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:51.947+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:52.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:52.253+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:52.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:52.948+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:53.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:53.231+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:53.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:53.994+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:54.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:54.269+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:55.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:55.010+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:55.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:55.253+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:55.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:55.982+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:56.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:56.293+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:57.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:57.015+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:57.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:57.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:58.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:58.026+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:58.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:58.293+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:59.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:59.006+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:59.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:16:59.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:16:59.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:16:59.962+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:00.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:00.289+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:00.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:00.915+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:01.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:01.300+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:01.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:01.871+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:02.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:02.293+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:02.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:02.879+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:03.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:03.301+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:03.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:03.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:04.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:04.293+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:04.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:04.885+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:05.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:05.326+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:05.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:05.921+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:06.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:06.372+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:06.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:06.933+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:07.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:07.389+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:07.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:07.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:08.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:08.380+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:08.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:08.849+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:09.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:09.399+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:09.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:09.876+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:10.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:10.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:10.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:10.916+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:11.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:11.388+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:11.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:11.906+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:12.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:12.381+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:12.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:12.895+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:13.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:13.420+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:13.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:13.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:14.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:14.404+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:14.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:14.890+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:15.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:15.379+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:15.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:15.923+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:16.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:16.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:16.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:16.875+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:17.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:17.327+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:17.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:17.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:18.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:18.318+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:18.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:18.846+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:19.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:19.342+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:19.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:19.851+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:20.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:20.334+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:20.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:20.856+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:21.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:21.333+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:21.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:21.869+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:22.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:22.375+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:22.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:22.876+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:23.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:23.410+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:23.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:23.872+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:24.443 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:24.443+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:24.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:24.842+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:25.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:25.476+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:25.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:25.875+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:26.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:26.485+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:26.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:26.844+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:27.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:27.465+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:27.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:27.863+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:28.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:28.458+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:28.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:28.858+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:29.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:29.450+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:29.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:29.850+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:30.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:30.455+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:30.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:30.862+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:31.423 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:31.422+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:31.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:31.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:32.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:32.409+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:32.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:32.909+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:33.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:33.384+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:33.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:33.922+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:34.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:34.409+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:34.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:34.910+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:35.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:35.414+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:35.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:35.951+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:36.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:36.375+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:36.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:36.934+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:37.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:37.396+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:37.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:37.964+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:38.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:38.372+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:38.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:38.990+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:39.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:39.344+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:40.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:40.002+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:40.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:40.336+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:40.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:40.970+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:41.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:41.351+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:41.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:41.934+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:42.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:42.398+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:42.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:42.967+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:43.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:43.442+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:43.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:43.956+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:44.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:44.405+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:44.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:44.990+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:45.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:45.380+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:46.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:46.027+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:46.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:46.367+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:47.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:47.072+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:47.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:47.343+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:48.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:48.077+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:48.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:48.382+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:49.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:49.076+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:49.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:49.401+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:50.064 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:50.064+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:50.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:50.449+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:51.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:51.020+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:51.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:51.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:52.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:52.062+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:52.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:52.479+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:53.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:53.070+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:53.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:53.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:54.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:54.041+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:54.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:54.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:55.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:55.023+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:55.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:55.546+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:56.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:56.052+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:56.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:56.553+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:57.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:57.032+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:57.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:57.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:58.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:58.044+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:58.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:58.563+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:59.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:17:59.017+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:17:59.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:17:59.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:00.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:00.028+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:00.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:00.560+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:01.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:01.011+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:01.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:01.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:02.003 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:02.003+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:02.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:02.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:03.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:03.000+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:03.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:03.609+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:04.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:04.018+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:04.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:04.599+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:05.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:05.038+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:05.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:05.648+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:06.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:06.062+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:06.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:06.683+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:07.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:07.104+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:07.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:07.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:08.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:08.064+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:08.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:08.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:09.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:09.079+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:09.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:09.648+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:10.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:10.110+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:10.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:10.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:11.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:11.101+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:11.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:11.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:12.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:12.059+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:12.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:12.573+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:13.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:13.091+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:13.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:13.596+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:14.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:14.136+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:14.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:14.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:15.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:15.142+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:15.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:15.590+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:16.164 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:16.164+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:16.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:16.558+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:17.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:17.157+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:17.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:17.519+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:18.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:18.181+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:18.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:18.495+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:19.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:19.214+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:19.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:19.491+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:20.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:20.202+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:20.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:20.485+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:21.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:21.180+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:21.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:21.533+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:22.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:22.137+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:22.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:22.537+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:23.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:23.117+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:23.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:23.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:24.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:24.090+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:24.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:24.499+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:25.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:25.085+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:25.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:25.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:26.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:26.083+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:26.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:26.539+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:27.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:27.119+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:27.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:27.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:28.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:28.070+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:28.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:28.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:29.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:29.040+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:29.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:29.534+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:30.015 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:30.016+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:30.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:30.489+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:31.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:31.046+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:31.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:31.473+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:32.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:32.084+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:32.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:32.454+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:33.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:33.058+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:33.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:33.411+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:34.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:34.105+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:34.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:34.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:35.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:35.140+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:35.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:35.375+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:36.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:36.187+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:36.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:36.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:37.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:37.205+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:37.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:37.406+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:38.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:38.212+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:38.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:38.399+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:39.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:39.177+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:39.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:39.410+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:40.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:40.178+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:40.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:40.428+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:41.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:41.140+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:41.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:41.477+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:42.128 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:42.129+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:42.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:42.494+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:43.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:43.178+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:43.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:43.454+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:44.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:44.165+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:44.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:44.426+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:45.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:45.196+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:45.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:45.437+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:46.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:46.195+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:46.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:46.424+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:47.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:47.160+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:47.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:47.391+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:48.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:48.191+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:48.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:48.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:49.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:49.176+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:49.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:49.376+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:50.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:50.204+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:50.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:50.327+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:51.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:51.226+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:51.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:51.286+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:52.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:52.267+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:52.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:52.267+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:53.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:53.271+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:53.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:53.290+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:54.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:54.231+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:54.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:54.328+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:55.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:55.257+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:55.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:55.378+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:56.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:56.295+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:56.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:56.409+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:57.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:57.292+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:57.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:57.411+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:58.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:58.307+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:58.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:58.426+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:59.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:18:59.302+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:18:59.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:18:59.460+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:00.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:00.351+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:00.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:00.452+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:01.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:01.396+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:01.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:01.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:02.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:02.383+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:02.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:02.435+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:03.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:03.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:03.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:03.444+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:04.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:04.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:04.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:04.473+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:05.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:05.432+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:05.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:05.496+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:06.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:06.475+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:06.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:06.506+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:07.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:07.487+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:07.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:07.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:08.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:08.508+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:08.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:08.518+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:09.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:09.473+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:09.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:09.510+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:10.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:10.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:10.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:10.498+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:11.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:11.453+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:11.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:11.469+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:12.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:12.445+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:12.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:12.459+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:13.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:13.462+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:13.495 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:13.495+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:14.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:14.461+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:14.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:14.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:15.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:15.442+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:15.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:15.564+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:16.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:16.489+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:16.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:16.609+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:17.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:17.495+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:17.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:17.644+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:18.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:18.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:18.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:18.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:19.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:19.502+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:19.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:19.596+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:20.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:20.516+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:20.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:20.596+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:21.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:21.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:21.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:21.578+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:22.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:22.516+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:22.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:22.541+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:23.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:23.494+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:23.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:23.522+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:24.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:24.485+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:24.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:24.550+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:25.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:25.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:25.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:25.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:26.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:26.566+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:26.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:26.601+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:27.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:27.523+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:27.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:27.582+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:28.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:28.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:28.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:28.551+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:29.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:29.518+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:29.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:29.583+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:30.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:30.488+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:30.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:30.606+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:31.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:31.486+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:31.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:31.567+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:32.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:32.520+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:32.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:32.556+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:33.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:33.492+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:33.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:33.564+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:34.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:34.479+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:34.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:34.538+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:35.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:35.474+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:35.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:35.518+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:36.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:36.439+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:36.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:36.539+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:37.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:37.399+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:37.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:37.527+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:38.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:38.386+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:38.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:38.508+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:39.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:39.424+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:39.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:39.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:40.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:40.395+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:40.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:40.499+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:41.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:41.369+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:41.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:41.513+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:42.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:42.412+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:42.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:42.500+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:43.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:43.391+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:43.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:43.510+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:44.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:44.361+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:44.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:44.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:45.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:45.331+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:45.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:45.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:46.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:46.371+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:46.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:46.506+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:47.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:47.323+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:47.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:47.523+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:48.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:48.280+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:48.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:48.560+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:49.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:49.288+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:49.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:49.542+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:50.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:50.297+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:50.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:50.557+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:51.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:51.327+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:51.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:51.520+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:52.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:52.290+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:52.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:52.476+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:53.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:53.324+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:53.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:53.472+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:54.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:54.329+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:54.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:54.505+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:55.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:55.363+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:55.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:55.480+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:56.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:56.322+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:56.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:56.489+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:57.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:57.300+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:57.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:57.484+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:58.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:58.343+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:58.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:58.458+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:59.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:19:59.369+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:19:59.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:19:59.470+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:00.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:00.402+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:00.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:00.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:01.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:01.353+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:01.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:01.550+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:02.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:02.326+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:02.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:02.513+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:03.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:03.316+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:03.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:03.520+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:04.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:04.296+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:04.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:04.548+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:05.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:05.345+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:05.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:05.501+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:06.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:06.387+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:06.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:06.480+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:07.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:07.409+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:07.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:07.431+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:08.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:08.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:08.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:08.455+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:09.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:09.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:09.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:09.418+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:10.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:10.381+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:10.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:10.410+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:11.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:11.354+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:11.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:11.448+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:12.395 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:12.396+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:12.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:12.467+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:13.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:13.375+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:13.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:13.499+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:14.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:14.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:14.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:14.477+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:15.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:15.427+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:15.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:15.489+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:16.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:16.443+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:16.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:16.538+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:17.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:17.455+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:17.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:17.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:18.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:18.464+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:18.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:18.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:19.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:19.433+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:19.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:19.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:20.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:20.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:20.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:20.564+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:21.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:21.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:21.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:21.611+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:22.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:22.473+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:22.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:22.564+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:23.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:23.428+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:23.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:23.527+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:24.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:24.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:24.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:24.489+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:25.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:25.362+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:25.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:25.530+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:26.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:26.388+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:26.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:26.537+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:27.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:27.393+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:27.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:27.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:28.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:28.393+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:28.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:28.526+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:29.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:29.379+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:29.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:29.543+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:30.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:30.367+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:30.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:30.581+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:31.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:31.355+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:31.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:31.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:32.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:32.326+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:32.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:32.528+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:33.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:33.336+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:33.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:33.515+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:34.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:34.351+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:34.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:34.545+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:35.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:35.362+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:35.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:35.567+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:36.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:36.376+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:36.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:36.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:37.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:37.354+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:37.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:37.516+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:38.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:38.323+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:38.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:38.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:39.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:39.332+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:39.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:39.503+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:40.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:40.311+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:40.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:40.521+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:41.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:41.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:41.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:41.536+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:42.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:42.328+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:42.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:42.517+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:43.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:43.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:43.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:43.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:44.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:44.341+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:44.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:44.565+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:45.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:45.337+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:45.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:45.585+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:46.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:46.349+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:46.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:46.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:47.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:47.364+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:47.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:47.503+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:48.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:48.377+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:48.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:48.551+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:49.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:49.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:49.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:49.570+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:50.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:50.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:50.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:50.571+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:51.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:51.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:51.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:51.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:52.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:52.404+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:52.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:52.624+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:53.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:53.440+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:53.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:53.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:54.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:54.405+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:54.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:54.637+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:55.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:55.383+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:55.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:55.623+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:56.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:56.388+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:56.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:56.636+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:57.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:57.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:57.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:57.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:58.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:58.428+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:58.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:58.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:59.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:20:59.411+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:20:59.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:20:59.615+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:00.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:00.423+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:00.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:00.647+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:01.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:01.420+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:01.688 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:01.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:02.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:02.464+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:02.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:02.688+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:03.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:03.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:03.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:03.696+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:04.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:04.486+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:04.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:04.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:05.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:05.486+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:05.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:05.699+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:06.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:06.487+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:06.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:06.686+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:07.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:07.505+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:07.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:07.664+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:08.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:08.518+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:08.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:08.621+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:09.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:09.496+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:09.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:09.646+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:10.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:10.465+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:10.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:10.631+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:11.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:11.450+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:11.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:11.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:12.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:12.448+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:12.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:12.609+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:13.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:13.432+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:13.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:13.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:14.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:14.387+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:14.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:14.692+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:15.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:15.432+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:15.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:15.678+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:16.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:16.461+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:16.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:16.666+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:17.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:17.469+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:17.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:17.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:18.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:18.458+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:18.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:18.650+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:19.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:19.469+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:19.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:19.663+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:20.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:20.493+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:20.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:20.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:21.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:21.459+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:21.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:21.675+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:22.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:22.440+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:22.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:22.693+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:23.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:23.403+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:23.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:23.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:24.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:24.368+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:24.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:24.671+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:25.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:25.390+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:25.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:25.644+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:26.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:26.368+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:26.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:26.618+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:27.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:27.357+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:27.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:27.612+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:28.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:28.327+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:28.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:28.585+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:29.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:29.298+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:29.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:29.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:30.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:30.275+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:30.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:30.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:31.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:31.265+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:31.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:31.607+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:32.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:32.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:32.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:32.587+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:33.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:33.257+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:33.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:33.568+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:34.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:34.238+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:34.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:34.561+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:35.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:35.262+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:35.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:35.564+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:36.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:36.291+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:36.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:36.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:37.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:37.310+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:37.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:37.547+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:38.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:38.316+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:38.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:38.583+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:39.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:39.287+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:39.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:39.577+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:40.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:40.322+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:40.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:40.606+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:41.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:41.344+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:41.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:41.647+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:42.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:42.388+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:42.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:42.634+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:43.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:43.409+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:43.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:43.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:44.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:44.382+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:44.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:44.689+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:45.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:45.336+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:45.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:45.654+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:46.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:46.310+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:46.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:46.654+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:47.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:47.321+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:47.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:47.676+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:48.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:48.298+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:48.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:48.644+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:49.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:49.330+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:49.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:49.622+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:50.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:50.356+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:50.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:50.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:51.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:51.356+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:51.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:51.572+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:52.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:52.371+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:52.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:52.616+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:53.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:53.397+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:53.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:53.638+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:54.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:54.445+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:54.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:54.608+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:55.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:55.466+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:55.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:55.626+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:56.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:56.461+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:56.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:56.642+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:57.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:57.481+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:57.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:57.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:58.491 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:58.490+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:58.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:58.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:59.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:21:59.508+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:21:59.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:21:59.611+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:00.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:00.471+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:00.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:00.589+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:01.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:01.436+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:01.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:01.583+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:02.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:02.400+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:02.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:02.559+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:03.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:03.415+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:03.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:03.542+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:04.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:04.457+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:04.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:04.498+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:05.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:05.449+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:05.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:05.472+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:06.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:06.466+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:06.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:06.477+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:07.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:07.420+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:07.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:07.481+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:08.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:08.456+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:08.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:08.497+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:09.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:09.462+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:09.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:09.495+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:10.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:10.456+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:10.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:10.461+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:11.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:11.414+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:11.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:11.483+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:12.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:12.397+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:12.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:12.473+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:13.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:13.379+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:13.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:13.509+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:14.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:14.415+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:14.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:14.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:15.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:15.430+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:15.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:15.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:16.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:16.392+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:16.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:16.569+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:17.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:17.379+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:17.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:17.613+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:18.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:18.427+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:18.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:18.633+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:19.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:19.443+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:19.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:19.588+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:20.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:20.482+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:20.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:20.614+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:21.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:21.517+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:21.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:21.639+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:22.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:22.509+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:22.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:22.655+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:23.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:23.528+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:23.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:23.704+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:24.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:24.487+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:24.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:24.713+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:25.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:25.473+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:25.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:25.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:26.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:26.444+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:26.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:26.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:27.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:27.437+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:27.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:27.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:28.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:28.473+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:28.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:28.636+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:29.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:29.433+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:29.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:29.654+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:30.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:30.474+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:30.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:30.675+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:31.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:31.450+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:31.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:31.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:32.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:32.479+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:32.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:32.636+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:33.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:33.459+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:33.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:33.591+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:34.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:34.502+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:34.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:34.557+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:35.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:35.510+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:35.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:35.549+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:36.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:36.537+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:36.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:36.553+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:37.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:37.531+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:37.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:37.560+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:38.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:38.497+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:38.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:38.609+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:39.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:39.540+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:39.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:39.620+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:40.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:40.548+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:40.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:40.667+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:41.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:41.579+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:41.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:41.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:42.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:42.605+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:42.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:42.665+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:43.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:43.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:43.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:43.681+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:44.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:44.592+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:44.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:44.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:45.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:45.551+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:45.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:45.726+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:46.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:46.529+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:46.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:46.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:47.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:47.550+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:47.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:47.698+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:48.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:48.601+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:48.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:48.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:49.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:49.623+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:49.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:49.735+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:50.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:50.640+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:50.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:50.766+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:51.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:51.677+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:51.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:51.789+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:52.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:52.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:52.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:52.793+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:53.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:53.705+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:53.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:53.771+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:54.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:54.716+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:54.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:54.744+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:55.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:55.755+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:55.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:55.773+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:56.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:56.788+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:56.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:56.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:57.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:57.760+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:57.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:57.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:58.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:58.793+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:58.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:58.843+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:59.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:22:59.816+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:22:59.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:22:59.840+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:00.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:00.842+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:00.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:00.867+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:01.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:01.821+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:01.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:01.878+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:02.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:02.844+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:02.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:02.884+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:03.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:03.875+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:03.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:03.893+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:04.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:04.879+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:04.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:04.936+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:05.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:05.886+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:05.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:05.920+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:06.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:06.898+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:06.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:06.959+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:07.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:07.855+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:07.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:07.962+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:08.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:08.830+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:08.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:08.918+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:09.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:09.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:09.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:09.900+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:10.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:10.787+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:10.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:10.886+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:11.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:11.757+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:11.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:11.929+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:12.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:12.748+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:12.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:12.894+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:13.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:13.793+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:13.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:13.864+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:14.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:14.746+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:14.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:14.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:15.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:15.773+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:15.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:15.809+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:16.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:16.748+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:16.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:16.795+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:17.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:17.790+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:17.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:17.814+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:18.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:18.796+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:19.102 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:18.837+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:19.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:19.752+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:19.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:19.839+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:20.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:20.707+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:20.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:20.840+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:21.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:21.668+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:21.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:21.822+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:22.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:22.653+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:22.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:22.782+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:23.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:23.686+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:23.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:23.811+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:24.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:24.684+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:24.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:24.819+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:25.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:25.712+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:25.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:25.781+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:26.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:26.750+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:26.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:26.764+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:27.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:27.771+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:27.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:27.774+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:28.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:28.736+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:28.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:28.820+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:29.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:29.726+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:29.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:29.824+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:30.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:30.700+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:30.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:30.819+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:31.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:31.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:31.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:31.808+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:32.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:32.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:32.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:32.780+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:33.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:33.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:33.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:33.827+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:34.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:34.685+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:34.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:34.799+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:35.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:35.712+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:35.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:35.823+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:36.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:36.692+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:36.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:36.784+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:37.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:37.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:37.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:37.752+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:38.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:38.704+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:38.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:38.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:39.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:39.701+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:39.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:39.726+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:40.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:40.653+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:40.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:40.768+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:41.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:41.613+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:41.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:41.804+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:42.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:42.588+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:42.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:42.778+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:43.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:43.608+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:43.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:43.731+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:44.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:44.616+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:44.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:44.697+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:45.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:45.606+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:45.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:45.694+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:46.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:46.646+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:46.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:46.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:47.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:47.624+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:47.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:47.704+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:48.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:48.593+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:48.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:48.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:49.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:49.619+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:49.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:49.670+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:50.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:50.645+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:50.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:50.647+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:51.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:51.632+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:51.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:51.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:52.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:52.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:52.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:52.673+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:53.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:53.596+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:53.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:53.675+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:54.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:54.612+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:54.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:54.701+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:55.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:55.629+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:55.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:55.696+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:56.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:56.620+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:56.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:56.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:57.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:57.623+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:57.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:57.634+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:58.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:58.615+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:58.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:58.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:59.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:23:59.617+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:23:59.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:23:59.646+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:00.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:00.575+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:00.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:00.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:01.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:01.605+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:01.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:01.647+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:02.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:02.620+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:02.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:02.651+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:03.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:03.621+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:03.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:03.624+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:04.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:04.595+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:04.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:04.618+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:05.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:05.590+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:05.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:05.615+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:06.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:06.605+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:06.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:06.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:07.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:07.610+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:07.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:07.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:08.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:08.653+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:08.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:08.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:09.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:09.670+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:09.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:09.735+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:10.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:10.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:10.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:10.710+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:11.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:11.695+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:11.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:11.701+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:12.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:12.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:12.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:12.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:13.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:13.671+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:13.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:13.689+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:14.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:14.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:14.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:14.715+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:15.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:15.687+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:15.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:15.716+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:16.688 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:16.688+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:16.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:16.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:17.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:17.647+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:17.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:17.667+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:18.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:18.603+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:18.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:18.648+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:19.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:19.621+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:19.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:19.660+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:20.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:20.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:20.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:20.694+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:21.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:21.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:21.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:21.678+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:22.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:22.686+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:22.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:22.686+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:23.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:23.650+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:23.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:23.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:24.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:24.682+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:24.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:24.765+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:25.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:25.696+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:25.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:25.790+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:26.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:26.672+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:26.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:26.767+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:27.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:27.701+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:27.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:27.727+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:28.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:28.680+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:28.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:28.683+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:29.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:29.634+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:29.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:29.650+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:30.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:30.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:30.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:30.683+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:31.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:31.650+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:31.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:31.712+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:32.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:32.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:32.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:32.706+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:33.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:33.689+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:33.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:33.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:34.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:34.711+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:34.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:34.722+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:35.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:35.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:35.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:35.708+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:36.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:36.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:36.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:36.722+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:37.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:37.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:37.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:37.743+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:38.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:38.669+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:38.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:38.789+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:39.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:39.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:39.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:39.785+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:40.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:40.655+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:40.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:40.810+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:41.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:41.668+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:41.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:41.795+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:42.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:42.641+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:42.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:42.770+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:43.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:43.665+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:43.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:43.724+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:44.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:44.668+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:44.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:44.701+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:45.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:45.623+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:45.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:45.690+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:46.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:46.642+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:46.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:46.682+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:47.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:47.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:47.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:47.674+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:48.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:48.691+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:48.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:48.704+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:49.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:49.659+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:49.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:49.734+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:50.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:50.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:50.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:50.748+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:51.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:51.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:51.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:51.727+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:52.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:52.707+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:52.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:52.727+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:53.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:53.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:53.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:53.759+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:54.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:54.672+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:54.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:54.779+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:55.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:55.709+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:55.764 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:55.764+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:56.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:56.747+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:56.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:56.809+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:57.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:57.763+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:57.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:57.837+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:58.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:58.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:58.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:58.860+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:59.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:24:59.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:24:59.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:24:59.874+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:00.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:00.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:00.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:00.913+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:01.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:01.712+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:01.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:01.880+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:02.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:02.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:02.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:02.869+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:03.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:03.761+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:03.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:03.853+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:04.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:04.731+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:04.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:04.901+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:05.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:05.721+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:05.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:05.859+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:06.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:06.727+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:06.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:06.856+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:07.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:07.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:07.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:07.876+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:08.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:08.738+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:08.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:08.916+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:09.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:09.756+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:09.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:09.903+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:10.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:10.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:10.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:10.895+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:11.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:11.785+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:11.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:11.891+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:12.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:12.771+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:12.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:12.871+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:13.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:13.734+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:13.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:13.918+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:14.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:14.752+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:14.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:14.910+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:15.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:15.720+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:15.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:15.871+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:16.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:16.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:16.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:16.875+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:17.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:17.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:17.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:17.877+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:18.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:18.678+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:18.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:18.919+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:19.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:19.700+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:19.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:19.966+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:20.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:20.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:20.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:20.926+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:21.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:21.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:21.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:21.882+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:22.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:22.664+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:22.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:22.857+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:23.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:23.656+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:23.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:23.869+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:24.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:24.619+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:24.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:24.828+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:25.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:25.632+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:25.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:25.847+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:26.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:26.641+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:26.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:26.891+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:27.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:27.644+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:27.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:27.856+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:28.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:28.658+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:28.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:28.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:29.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:29.694+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:29.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:29.862+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:30.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:30.683+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:30.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:30.890+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:31.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:31.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:31.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:31.841+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:32.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:32.656+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:32.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:32.851+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:33.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:33.693+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:33.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:33.837+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:34.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:34.737+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:34.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:34.816+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:35.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:35.719+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:35.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:35.787+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:36.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:36.708+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:36.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:36.762+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:37.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:37.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:37.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:37.803+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:38.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:38.738+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:38.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:38.826+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:39.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:39.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:39.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:39.807+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:40.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:40.730+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:40.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:40.766+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:41.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:41.732+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:41.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:41.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:42.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:42.764+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:42.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:42.765+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:43.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:43.800+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:43.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:43.807+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:44.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:44.761+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:44.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:44.806+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:45.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:45.771+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:45.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:45.848+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:46.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:46.767+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:46.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:46.835+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:47.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:47.730+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:47.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:47.866+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:48.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:48.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:48.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:48.916+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:49.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:49.717+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:49.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:49.948+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:50.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:50.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:50.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:50.980+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:51.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:51.726+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:51.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:51.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:52.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:52.702+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:52.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:52.978+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:53.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:53.740+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:53.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:53.984+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:54.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:54.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:54.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:54.968+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:55.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:55.673+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:55.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:55.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:56.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:56.722+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:57.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:57.035+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:57.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:57.758+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:57.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:57.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:58.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:58.757+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:59.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:59.031+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:59.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:25:59.735+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:25:59.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:25:59.991+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:00.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:00.774+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:00.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:00.958+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:01.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:01.784+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:01.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:01.977+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:02.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:02.781+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:02.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:02.963+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:03.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:03.749+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:03.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:03.983+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:04.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:04.738+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:04.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:04.971+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:05.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:05.711+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:05.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:05.957+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:06.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:06.671+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:06.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:06.996+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:07.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:07.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:07.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:07.965+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:08.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:08.634+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:08.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:08.958+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:09.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:09.660+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:09.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:09.919+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:10.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:10.638+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:10.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:10.908+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:11.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:11.607+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:11.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:11.866+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:12.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:12.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:12.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:12.843+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:13.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:13.643+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:13.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:13.821+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:14.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:14.672+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:14.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:14.805+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:15.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:15.640+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:15.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:15.819+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:16.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:16.631+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:16.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:16.838+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:17.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:17.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:17.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:17.845+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:18.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:18.635+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:18.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:18.889+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:19.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:19.622+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:19.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:19.939+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:20.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:20.629+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:20.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:20.986+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:21.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:21.636+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:21.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:21.976+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:22.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:22.681+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:22.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:22.961+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:23.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:23.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:23.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:23.918+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:24.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:24.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:24.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:24.885+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:25.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:25.720+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:25.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:25.892+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:26.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:26.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:26.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:26.926+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:27.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:27.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:27.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:27.964+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:28.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:28.718+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:28.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:28.916+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:29.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:29.690+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:29.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:29.892+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:30.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:30.683+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:30.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:30.924+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:31.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:31.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:31.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:31.941+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:32.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:32.702+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:32.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:32.935+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:33.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:33.723+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:33.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:33.966+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:34.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:34.678+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:34.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:34.965+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:35.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:35.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:36.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:36.015+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:36.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:36.744+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:36.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:36.999+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:37.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:37.780+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:38.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:38.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:38.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:38.760+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:39.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:39.007+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:39.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:39.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:40.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:40.017+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:40.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:40.743+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:40.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:40.985+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:41.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:41.750+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:42.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:42.033+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:42.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:42.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:43.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:43.069+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:43.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:43.753+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:44.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:44.078+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:44.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:44.797+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:45.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:45.085+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:45.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:45.773+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:46.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:46.103+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:46.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:46.754+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:47.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:47.118+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:47.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:47.776+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:48.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:48.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:48.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:48.745+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:49.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:49.123+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:49.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:49.731+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:50.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:50.110+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:50.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:50.710+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:51.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:51.107+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:51.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:51.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:52.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:52.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:52.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:52.659+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:53.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:53.117+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:53.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:53.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:54.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:54.098+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:54.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:54.714+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:55.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:55.060+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:55.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:55.716+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:56.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:56.088+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:56.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:56.668+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:57.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:57.070+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:57.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:57.624+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:58.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:58.098+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:58.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:58.651+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:59.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:26:59.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:26:59.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:26:59.660+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:00.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:00.041+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:00.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:00.676+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:01.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:01.024+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:01.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:01.715+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:02.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:02.040+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:02.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:02.674+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:03.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:03.054+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:03.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:03.649+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:04.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:04.082+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:04.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:04.630+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:05.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:05.091+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:05.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:05.654+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:06.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:06.062+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:06.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:06.661+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:07.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:07.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:07.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:07.688+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:08.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:08.059+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:08.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:08.648+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:09.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:09.037+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:09.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:09.604+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:10.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:10.001+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:10.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:10.591+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:10.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:10.994+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:11.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:11.588+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:11.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:11.986+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:12.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:12.627+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:12.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:12.966+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:13.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:13.677+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:14.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:14.012+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:14.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:14.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:15.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:15.024+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:15.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:15.682+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:15.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:15.976+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:16.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:16.728+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:16.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:16.978+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:17.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:17.703+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:17.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:17.962+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:18.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:18.705+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:18.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:18.998+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:19.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:19.663+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:20.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:20.010+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:20.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:20.679+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:21.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:21.046+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:21.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:21.657+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:22.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:22.056+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:22.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:22.615+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:23.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:23.011+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:23.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:23.569+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:23.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:23.987+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:24.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:24.551+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:25.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:25.024+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:25.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:25.577+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:26.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:26.064+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:26.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:26.628+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:27.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:27.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:27.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:27.641+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:28.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:28.007+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:28.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:28.685+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:29.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:29.052+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:29.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:29.725+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:30.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:30.090+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:30.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:30.713+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:31.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:31.070+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:31.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:31.695+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:32.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:32.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:32.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:32.699+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:33.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:33.034+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:33.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:33.729+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:34.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:34.079+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:34.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:34.742+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:35.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:35.059+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:35.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:35.766+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:36.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:36.044+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:36.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:36.763+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:37.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:36.999+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:37.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:37.736+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:38.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:38.001+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:38.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:38.774+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:39.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:39.049+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:39.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:39.780+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:40.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:40.068+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:40.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:40.782+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:41.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:41.112+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:41.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:41.817+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:42.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:42.109+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:42.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:42.787+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:43.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:43.124+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:43.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:43.747+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:44.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:44.148+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:44.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:44.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:45.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:45.102+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:45.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:45.800+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:46.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:46.151+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:46.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:46.796+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:47.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:47.153+0000 7f1738067700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:47.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:47.841+0000 7f305c966700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:48.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:48.123+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:48.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:48.827+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:49.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:49.149+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:49.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:49.836+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:50.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:50.146+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:50.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:50.804+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:51.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:51.135+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:51.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:51.843+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:52.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:52.110+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:52.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:52.873+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:53.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:53.149+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:53.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:53.852+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:54.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:54.104+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:54.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:54.848+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:55.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:55.089+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:55.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:55.835+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:56.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:56.124+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:56.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:56.791+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:57.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:57.101+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:57.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:57.785+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:58.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:58.054+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:58.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:58.806+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:59.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:27:59.023+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:27:59.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:27:59.814+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:00.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:00.055+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:00.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:00.806+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:01.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:01.085+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:01.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:01.827+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:02.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:02.099+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:02.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:02.796+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:03.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:03.080+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:03.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:03.786+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:04.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:04.045+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:04.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:04.788+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:05.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:05.031+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:05.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:05.767+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:06.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:06.026+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:06.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:06.734+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:07.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:07.047+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:07.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:07.702+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:08.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:08.032+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:08.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:08.736+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:09.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:09.073+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:09.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:09.699+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:10.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:10.114+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:10.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:10.698+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:11.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:11.101+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:11.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:11.682+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:12.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:12.063+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:12.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:12.637+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:13.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:13.045+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:13.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:13.600+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:13.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:13.996+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:14.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:14.603+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:15.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:15.013+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:15.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:15.596+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:16.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:16.042+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:16.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:16.643+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:17.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:17.039+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:17.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:17.665+0000 7f305c966700 -1 osd.0 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:18.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:18.033+0000 7f1738067700 -1 osd.1 161 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:18.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:18.680+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:19.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:19.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:19.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:19.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:20.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:20.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:20.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:20.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:20.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:20.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:21.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:21.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:22.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:22.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:22.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:22.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:23.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:23.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:23.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:23.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:24.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:24.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:24.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:24.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:25.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:25.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:25.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:25.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:26.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:26.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:26.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:26.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:27.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:27.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:27.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:27.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:28.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:28.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:28.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:28.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:29.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:29.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:29.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:29.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:30.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:30.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:30.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:30.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:31.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:31.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:31.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:31.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:32.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:32.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:32.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:32.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:33.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:33.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:33.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:33.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:34.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:34.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:34.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:34.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:35.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:35.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:35.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:35.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:35.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:35.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:36.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:36.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:36.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:36.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:37.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:37.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:37.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:37.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:38.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:38.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:38.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:38.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:39.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:39.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:40.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:40.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:40.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:40.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:40.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:40.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:41.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:41.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:41.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:41.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:42.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:42.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:42.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:42.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:43.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:43.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:43.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:43.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:44.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:44.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:44.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:44.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:45.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:45.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:45.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:45.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:46.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:46.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:46.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:46.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:47.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:47.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:47.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:47.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:48.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:48.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:48.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:48.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:49.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:49.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:49.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:49.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:50.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:50.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:50.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:50.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:51.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:51.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:51.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:51.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:52.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:52.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:52.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:52.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:53.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:53.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:53.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:53.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:54.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:54.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:54.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:54.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:55.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:55.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:55.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:55.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:56.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:56.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:56.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:56.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:57.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:57.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:57.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:57.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:58.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:58.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:58.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:58.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:59.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:28:59.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:28:59.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:28:59.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:00.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:00.912+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:00.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:00.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:01.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:01.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:01.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:01.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:02.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:02.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:02.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:02.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:03.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:03.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:03.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:03.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:04.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:04.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:04.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:04.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:05.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:05.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:05.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:05.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:06.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:06.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:06.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:06.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:07.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:07.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:07.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:07.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:08.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:08.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:08.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:08.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:09.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:09.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:09.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:09.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:10.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:10.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:10.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:10.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:11.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:11.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:11.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:11.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:12.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:12.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:12.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:12.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:13.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:13.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:13.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:13.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:14.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:14.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:14.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:14.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:15.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:15.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:15.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:15.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:16.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:16.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:16.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:16.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:17.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:17.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:17.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:17.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:18.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:18.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:19.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:19.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:19.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:19.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:20.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:20.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:20.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:20.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:21.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:21.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:21.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:21.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:21.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:21.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:22.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:22.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:22.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:22.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:23.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:23.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:24.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:24.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:24.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:24.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:25.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:25.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:25.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:25.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:26.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:26.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:26.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:26.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:26.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:26.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:27.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:27.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:27.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:27.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:28.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:28.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:28.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:28.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:29.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:29.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:30.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:30.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:30.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:30.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:30.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:30.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:31.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:31.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:32.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:32.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:32.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:32.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:32.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:32.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:33.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:33.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:33.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:33.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:34.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:34.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:34.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:34.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:35.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:35.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:35.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:35.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:36.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:36.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:36.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:36.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:37.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:37.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:38.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:38.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:38.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:38.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:39.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:39.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:39.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:39.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:40.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:40.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:40.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:40.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:41.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:41.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:41.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:41.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:42.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:42.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:42.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:42.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:43.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:43.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:43.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:43.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:44.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:44.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:44.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:44.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:45.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:45.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:45.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:45.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:45.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:45.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:46.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:46.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:46.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:46.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:47.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:47.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:48.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:48.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:48.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:48.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:49.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:49.001+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:49.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:49.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:50.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:50.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:50.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:50.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:51.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:51.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:51.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:51.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:52.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:52.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:52.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:52.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:53.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:53.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:53.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:53.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:54.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:54.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:54.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:54.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:55.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:55.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:55.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:55.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:56.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:56.155+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:56.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:56.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:57.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:57.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:57.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:57.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:58.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:58.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:58.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:58.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:59.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:29:59.101+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:29:59.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:29:59.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:00.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:00.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:00.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:00.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:01.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:01.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:01.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:01.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:02.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:02.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:02.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:02.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:03.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:03.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:03.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:03.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:04.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:04.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:04.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:04.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:05.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:05.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:05.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:05.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:06.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:06.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:06.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:06.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:07.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:07.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:07.573 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T19:30:07.573+0000 7ffb851f6700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6800/86743,v1:172.21.15.143:6801/86743] conn(0x7ffb6400a020 msgr2=0x7ffb6c046e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T19:30:07.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:07.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:08.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:08.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:08.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:08.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:09.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:09.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:09.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:09.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:10.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:10.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:10.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:10.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:11.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:11.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:11.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:11.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:12.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:12.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:12.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:12.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:13.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:13.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:13.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:13.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:14.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:14.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:14.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:14.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:15.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:15.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:15.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:15.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:16.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:16.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:16.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:16.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:17.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:17.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:17.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:17.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:18.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:18.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:18.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:18.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:19.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:19.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:19.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:19.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:20.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:20.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:20.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:20.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:21.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:21.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:21.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:21.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:22.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:22.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:22.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:22.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:23.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:23.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:23.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:23.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:24.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:24.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:24.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:24.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:25.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:25.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:25.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:25.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:26.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:26.207+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:26.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:26.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:27.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:27.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:27.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:27.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:28.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:28.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:28.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:28.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:29.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:29.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:29.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:29.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:30.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:30.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:30.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:30.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:31.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:31.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:31.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:31.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:32.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:32.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:32.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:32.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:33.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:33.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:33.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:33.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:34.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:34.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:34.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:34.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:35.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:35.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:35.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:35.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:36.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:36.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:36.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:36.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:37.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:37.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:37.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:37.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:38.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:38.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:38.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:38.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:39.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:39.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:39.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:39.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:40.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:40.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:40.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:40.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:41.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:41.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:41.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:41.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:42.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:42.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:42.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:42.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:43.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:43.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:43.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:43.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:44.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:44.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:44.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:44.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:45.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:45.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:45.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:45.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:45.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:45.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:46.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:46.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:47.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:47.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:47.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:47.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:47.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:47.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:48.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:48.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:49.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:49.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:49.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:49.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:50.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:50.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:50.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:50.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:50.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:50.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:51.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:51.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:51.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:51.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:52.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:52.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:52.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:52.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:53.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:53.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:53.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:53.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:54.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:54.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:54.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:54.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:55.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:55.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:55.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:55.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:56.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:56.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:56.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:56.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:57.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:57.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:57.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:57.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:58.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:58.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:58.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:58.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:59.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:30:59.761+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:30:59.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:30:59.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:00.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:00.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:00.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:00.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:01.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:01.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:01.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:01.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:02.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:02.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:02.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:02.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:03.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:03.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:03.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:03.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:04.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:04.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:04.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:04.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:05.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:05.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:05.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:05.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:06.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:06.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:06.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:06.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:07.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:07.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:07.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:07.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:08.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:08.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:08.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:08.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:09.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:09.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:09.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:09.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:10.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:10.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:10.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:10.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:11.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:11.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:11.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:11.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:12.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:12.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:12.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:12.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:13.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:13.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:13.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:13.805+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:14.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:14.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:14.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:14.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:15.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:15.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:15.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:15.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:16.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:16.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:16.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:16.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:17.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:17.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:17.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:17.799+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:18.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:18.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:18.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:18.762+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:19.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:19.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:19.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:19.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:20.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:20.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:20.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:20.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:21.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:21.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:21.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:21.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:22.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:22.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:22.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:22.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:23.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:23.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:23.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:23.786+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:24.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:24.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:24.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:24.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:25.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:25.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:25.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:25.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:26.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:26.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:26.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:26.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:27.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:27.775+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:27.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:27.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:28.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:28.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:28.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:28.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:29.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:29.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:29.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:29.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:30.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:30.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:30.863 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:30.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:31.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:31.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:31.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:31.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:32.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:32.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:32.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:32.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:33.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:33.672+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:33.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:33.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:34.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:34.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:34.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:34.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:35.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:35.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:35.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:35.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:36.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:36.651+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:36.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:36.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:37.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:37.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:37.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:37.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:38.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:38.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:38.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:38.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:39.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:39.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:39.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:39.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:40.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:40.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:40.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:40.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:41.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:41.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:41.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:41.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:42.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:42.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:42.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:42.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:43.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:43.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:43.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:43.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:44.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:44.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:44.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:44.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:45.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:45.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:45.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:45.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:46.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:46.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:46.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:46.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:47.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:47.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:47.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:47.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:48.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:48.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:48.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:48.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:49.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:49.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:49.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:49.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:50.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:50.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:50.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:50.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:51.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:51.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:51.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:51.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:52.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:52.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:52.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:52.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:53.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:53.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:53.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:53.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:54.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:54.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:54.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:54.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:55.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:55.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:55.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:55.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:56.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:56.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:56.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:56.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:57.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:57.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:57.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:57.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:58.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:58.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:59.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:59.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:59.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:31:59.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:31:59.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:31:59.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:00.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:00.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:00.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:00.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:01.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:01.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:01.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:01.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:02.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:02.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:02.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:02.984+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:03.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:03.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:04.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:04.001+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:04.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:04.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:04.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:04.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:05.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:05.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:05.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:05.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:06.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:06.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:06.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:06.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:07.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:07.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:07.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:07.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:08.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:08.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:09.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:09.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:09.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:09.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:09.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:09.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:10.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:10.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:10.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:10.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:11.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:11.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:12.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:12.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:12.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:12.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:12.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:12.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:13.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:13.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:13.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:13.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:14.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:14.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:15.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:15.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:15.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:15.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:16.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:16.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:16.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:16.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:16.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:16.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:17.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:17.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:17.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:17.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:18.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:18.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:18.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:18.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:19.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:19.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:19.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:19.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:20.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:20.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:20.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:20.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:21.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:21.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:21.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:21.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:22.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:22.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:23.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:23.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:23.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:23.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:24.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:24.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:24.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:24.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:25.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:25.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:25.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:25.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:26.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:26.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:26.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:26.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:27.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:27.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:27.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:27.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:28.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:28.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:28.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:28.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:29.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:29.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:29.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:29.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:30.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:30.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:30.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:30.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:31.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:31.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:31.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:31.545+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:32.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:32.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:32.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:32.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:33.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:33.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:33.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:33.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:33.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:33.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:34.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:34.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:34.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:34.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:35.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:35.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:35.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:35.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:36.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:36.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:36.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:36.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:37.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:37.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:37.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:37.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:38.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:38.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:38.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:38.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:39.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:39.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:39.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:39.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:40.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:40.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:40.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:40.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:41.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:41.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:41.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:41.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:42.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:42.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:42.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:42.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:43.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:43.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:43.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:43.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:44.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:44.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:44.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:44.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:45.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:45.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:45.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:45.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:46.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:46.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:46.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:46.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:47.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:47.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:47.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:47.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:48.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:48.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:48.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:48.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:49.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:49.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:49.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:49.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:50.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:50.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:50.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:50.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:51.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:51.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:51.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:51.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:52.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:52.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:52.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:52.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:53.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:53.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:54.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:54.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:54.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:54.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:55.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:55.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:55.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:55.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:55.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:55.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:56.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:56.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:57.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:57.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:57.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:57.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:58.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:58.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:58.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:58.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:59.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:32:59.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:32:59.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:32:59.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:00.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:00.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:00.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:00.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:01.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:01.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:01.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:01.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:02.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:02.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:02.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:02.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:03.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:03.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:03.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:03.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:04.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:04.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:04.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:04.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:04.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:04.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:05.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:05.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:05.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:05.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:06.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:06.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:06.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:06.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:07.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:07.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:08.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:08.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:08.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:08.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:08.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:08.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:09.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:09.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:10.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:10.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:10.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:10.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:10.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:10.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:11.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:11.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:12.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:12.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:12.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:12.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:13.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:13.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:13.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:13.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:13.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:13.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:14.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:14.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:14.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:14.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:15.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:15.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:16.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:16.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:16.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:16.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:16.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:16.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:17.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:17.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:17.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:17.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:18.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:18.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:19.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:19.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:19.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:19.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:20.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:20.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:20.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:20.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:21.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:21.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:21.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:21.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:22.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:22.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:22.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:22.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:23.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:23.101+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:23.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:23.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:24.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:24.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:24.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:24.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:25.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:25.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:25.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:25.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:26.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:26.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:26.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:26.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:27.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:27.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:27.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:27.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:28.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:28.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:28.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:28.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:29.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:29.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:29.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:29.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:30.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:29.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:30.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:30.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:30.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:30.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:31.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:31.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:31.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:31.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:32.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:32.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:32.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:32.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:33.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:33.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:34.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:34.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:34.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:34.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:35.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:35.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:35.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:35.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:36.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:36.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:36.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:36.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:37.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:37.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:37.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:37.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:38.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:38.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:38.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:38.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:39.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:39.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:39.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:39.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:40.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:40.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:40.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:40.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:41.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:41.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:41.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:41.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:42.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:42.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:42.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:42.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:43.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:43.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:43.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:43.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:44.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:44.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:44.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:44.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:45.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:45.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:45.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:45.621+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:46.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:46.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:46.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:46.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:47.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:47.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:47.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:47.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:48.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:48.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:48.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:48.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:49.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:49.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:49.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:49.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:50.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:50.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:50.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:50.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:51.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:51.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:51.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:51.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:52.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:52.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:52.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:52.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:53.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:53.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:53.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:53.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:54.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:54.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:54.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:54.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:55.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:55.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:55.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:55.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:56.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:56.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:56.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:56.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:57.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:57.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:57.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:57.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:58.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:58.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:58.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:58.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:59.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:33:59.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:33:59.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:33:59.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:00.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:00.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:00.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:00.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:01.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:01.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:01.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:01.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:02.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:02.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:02.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:02.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:03.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:03.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:03.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:03.569+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:04.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:04.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:04.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:04.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:05.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:05.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:05.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:05.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:06.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:06.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:06.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:06.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:07.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:07.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:07.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:07.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:08.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:08.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:08.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:08.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:09.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:09.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:09.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:09.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:10.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:10.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:10.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:10.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:11.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:11.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:11.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:11.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:12.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:12.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:12.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:12.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:13.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:13.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:13.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:13.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:14.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:14.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:14.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:14.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:15.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:15.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:15.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:15.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:16.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:16.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:16.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:16.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:17.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:17.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:17.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:17.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:18.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:18.244+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:18.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:18.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:19.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:19.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:19.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:19.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:20.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:20.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:20.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:20.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:21.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:21.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:21.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:21.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:22.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:22.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:22.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:22.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:23.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:23.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:23.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:23.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:24.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:24.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:24.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:24.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:25.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:25.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:25.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:25.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:26.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:26.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:26.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:26.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:27.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:27.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:27.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:27.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:27.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:27.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:28.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:28.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:29.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:29.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:29.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:29.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:30.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:30.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:30.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:30.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:31.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:31.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:31.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:31.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:32.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:32.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:32.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:32.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:33.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:33.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:33.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:33.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:34.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:34.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:34.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:34.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:35.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:35.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:35.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:35.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:36.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:36.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:36.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:36.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:37.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:37.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:37.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:37.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:38.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:38.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:38.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:38.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:38.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:38.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:39.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:39.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:39.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:39.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:40.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:40.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:40.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:40.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:41.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:41.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:42.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:41.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:42.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:42.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:42.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:42.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:43.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:43.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:43.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:43.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:44.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:44.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:44.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:44.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:45.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:45.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:46.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:45.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:46.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:46.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:46.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:46.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:47.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:47.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:47.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:47.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:48.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:48.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:48.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:48.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:49.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:49.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:49.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:49.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:50.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:50.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:50.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:50.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:51.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:51.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:51.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:51.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:52.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:52.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:52.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:52.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:53.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:53.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:53.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:53.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:54.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:54.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:54.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:54.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:55.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:55.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:55.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:55.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:56.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:56.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:56.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:56.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:57.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:57.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:57.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:57.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:58.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:58.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:58.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:58.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:59.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:34:59.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:34:59.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:34:59.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:00.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:00.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:00.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:00.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:01.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:01.751+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:01.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:01.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:02.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:02.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:02.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:02.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:03.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:03.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:03.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:03.844+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:04.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:04.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:04.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:04.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:05.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:05.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:05.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:05.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:06.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:06.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:06.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:06.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:07.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:07.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:07.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:07.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:08.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:08.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:08.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:08.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:09.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:09.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:09.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:09.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:10.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:10.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:10.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:10.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:11.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:11.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:11.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:11.896+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:12.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:12.860+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:12.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:12.888+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:13.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:13.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:13.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:13.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:14.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:14.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:14.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:14.890+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:15.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:15.860+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:15.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:15.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:16.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:16.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:16.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:16.983+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:17.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:17.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:17.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:17.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:18.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:18.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:18.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:18.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:19.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:19.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:19.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:19.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:20.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:20.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:21.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:21.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:21.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:21.755+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:22.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:22.065+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:22.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:22.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:23.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:23.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:23.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:23.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:24.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:24.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:24.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:24.762+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:25.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:25.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:25.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:25.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:26.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:26.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:26.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:26.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:27.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:27.081+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:27.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:27.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:28.123 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:28.123+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:28.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:28.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:29.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:29.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:29.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:29.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:30.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:30.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:30.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:30.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:31.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:31.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:31.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:31.802+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:32.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:32.149+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:32.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:32.781+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:33.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:33.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:33.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:33.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:34.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:34.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:34.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:34.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:35.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:35.210+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:35.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:35.790+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:36.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:36.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:36.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:36.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:37.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:37.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:37.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:37.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:38.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:38.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:38.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:38.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:39.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:39.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:39.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:39.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:40.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:40.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:40.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:40.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:41.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:41.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:41.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:41.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:42.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:42.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:42.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:42.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:43.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:43.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:43.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:43.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:44.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:44.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:44.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:44.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:45.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:45.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:45.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:45.717+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:46.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:46.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:46.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:46.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:47.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:47.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:47.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:47.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:48.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:48.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:48.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:48.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:49.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:49.215+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:49.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:49.769+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:50.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:50.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:50.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:50.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:51.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:51.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:51.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:51.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:52.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:52.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:52.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:52.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:53.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:53.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:53.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:53.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:54.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:54.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:54.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:54.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:55.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:55.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:55.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:55.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:56.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:56.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:56.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:56.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:57.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:57.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:57.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:57.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:58.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:58.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:58.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:58.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:59.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:35:59.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:35:59.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:35:59.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:00.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:00.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:00.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:00.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:01.227 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:01.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:01.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:01.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:02.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:02.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:02.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:02.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:03.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:03.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:03.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:03.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:04.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:04.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:04.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:04.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:05.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:05.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:05.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:05.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:06.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:06.188+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:06.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:06.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:07.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:07.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:07.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:07.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:08.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:08.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:08.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:08.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:09.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:09.183+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:09.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:09.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:10.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:10.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:10.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:10.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:11.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:11.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:11.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:11.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:12.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:12.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:12.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:12.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:13.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:13.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:13.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:13.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:14.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:14.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:15.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:15.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:15.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:15.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:15.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:16.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:16.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:16.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:17.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:17.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:17.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:17.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:18.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:18.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:18.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:18.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:19.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:19.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:19.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:19.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:20.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:20.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:20.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:20.284+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:20.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:20.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:21.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:21.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:21.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:21.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:22.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:22.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:23.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:23.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:23.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:23.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:24.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:24.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:24.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:24.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:25.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:25.048+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:25.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:25.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:26.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:26.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:26.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:26.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:27.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:27.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:27.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:27.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:28.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:28.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:28.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:28.263+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:29.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:29.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:29.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:29.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:30.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:30.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:30.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:30.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:31.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:31.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:31.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:31.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:32.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:32.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:32.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:32.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:33.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:33.239+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:33.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:33.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:34.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:34.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:34.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:34.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:35.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:35.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:35.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:35.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:36.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:36.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:36.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:36.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:37.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:37.261+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:37.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:37.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:38.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:38.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:38.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:38.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:39.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:39.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:39.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:39.380+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:40.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:40.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:40.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:40.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:41.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:41.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:41.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:41.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:42.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:42.233+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:42.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:42.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:43.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:43.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:43.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:43.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:44.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:44.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:44.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:44.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:45.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:45.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:45.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:45.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:46.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:46.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:46.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:46.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:47.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:47.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:47.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:47.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:48.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:48.259+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:48.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:48.395+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:49.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:49.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:49.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:49.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:50.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:50.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:50.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:50.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:51.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:51.318+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:51.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:51.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:52.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:52.299+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:52.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:52.361+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:53.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:53.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:53.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:53.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:54.302 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:54.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:54.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:54.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:55.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:55.288+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:55.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:55.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:56.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:56.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:56.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:56.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:57.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:57.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:57.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:57.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:58.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:58.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:58.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:58.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:59.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:36:59.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:36:59.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:36:59.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:00.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:00.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:00.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:00.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:01.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:01.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:01.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:01.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:02.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:02.349+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:02.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:02.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:03.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:03.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:03.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:03.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:04.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:04.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:04.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:04.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:05.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:05.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:05.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:05.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:06.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:06.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:06.426 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:06.426+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:07.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:07.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:07.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:07.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:08.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:08.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:08.443 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:08.442+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:09.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:09.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:09.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:09.441+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:10.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:10.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:10.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:10.397+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:11.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:11.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:11.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:11.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:12.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:12.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:12.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:12.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:13.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:13.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:13.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:13.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:14.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:14.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:14.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:14.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:15.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:15.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:15.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:15.396+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:16.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:16.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:16.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:16.375+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:17.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:17.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:17.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:17.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:18.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:18.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:18.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:18.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:19.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:19.145+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:19.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:19.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:20.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:20.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:20.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:20.409+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:21.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:21.165+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:21.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:21.390+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:22.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:22.203+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:22.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:22.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:23.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:23.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:23.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:23.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:24.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:24.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:24.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:24.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:25.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:25.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:25.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:25.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:26.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:26.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:26.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:26.420+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:27.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:27.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:27.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:27.379+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:28.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:28.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:28.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:28.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:29.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:29.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:29.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:29.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:30.403 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:30.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:30.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:30.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:31.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:31.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:31.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:31.396+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:32.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:32.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:32.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:32.412+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:33.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:33.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:33.448 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:33.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:34.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:34.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:34.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:34.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:35.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:35.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:35.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:35.539+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:36.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:36.407+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:36.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:36.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:37.415 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:37.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:37.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:37.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:38.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:38.397+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:38.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:38.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:39.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:39.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:39.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:39.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:40.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:40.460+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:40.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:40.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:41.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:41.416+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:41.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:41.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:42.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:42.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:42.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:42.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:43.426 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:43.426+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:43.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:43.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:44.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:44.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:44.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:44.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:45.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:45.414+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:45.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:45.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:46.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:46.454+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:46.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:46.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:47.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:47.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:47.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:47.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:48.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:48.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:48.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:48.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:49.500 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:49.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:49.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:49.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:50.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:50.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:50.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:50.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:51.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:51.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:51.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:51.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:52.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:52.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:52.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:52.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:53.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:53.484+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:53.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:53.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:54.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:54.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:54.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:54.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:55.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:55.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:55.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:55.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:56.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:56.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:56.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:56.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:57.508 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:57.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:57.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:57.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:58.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:58.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:58.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:58.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:59.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:37:59.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:37:59.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:37:59.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:00.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:00.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:00.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:00.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:01.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:01.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:01.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:01.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:02.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:02.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:02.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:02.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:03.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:03.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:03.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:03.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:04.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:04.575+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:04.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:04.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:05.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:05.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:05.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:05.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:06.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:06.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:06.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:06.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:07.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:07.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:07.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:07.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:08.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:08.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:08.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:08.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:09.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:09.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:09.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:09.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:10.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:10.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:10.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:10.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:11.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:11.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:11.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:11.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:12.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:12.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:12.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:12.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:13.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:13.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:13.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:13.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:14.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:14.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:14.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:14.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:15.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:15.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:15.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:15.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:16.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:16.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:16.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:16.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:17.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:17.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:17.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:17.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:18.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:18.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:18.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:18.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:19.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:19.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:19.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:19.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:20.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:20.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:20.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:20.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:21.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:21.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:21.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:21.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:22.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:22.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:22.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:22.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:23.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:23.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:23.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:23.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:24.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:24.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:24.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:24.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:25.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:25.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:25.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:25.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:26.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:26.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:26.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:26.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:27.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:27.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:27.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:27.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:28.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:28.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:28.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:28.698+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:29.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:29.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:29.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:29.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:30.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:30.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:30.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:30.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:31.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:31.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:31.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:31.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:32.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:32.596+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:32.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:32.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:33.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:33.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:33.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:33.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:34.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:34.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:34.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:34.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:35.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:35.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:35.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:35.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:36.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:36.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:36.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:36.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:37.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:37.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:37.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:37.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:38.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:38.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:38.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:38.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:39.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:39.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:39.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:39.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:40.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:40.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:40.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:40.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:41.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:41.605+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:41.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:41.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:42.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:42.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:42.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:42.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:43.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:43.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:43.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:43.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:44.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:44.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:44.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:44.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:45.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:45.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:45.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:45.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:46.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:46.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:46.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:46.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:47.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:47.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:47.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:47.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:48.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:48.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:48.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:48.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:49.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:49.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:49.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:49.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:50.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:50.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:50.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:50.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:51.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:51.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:51.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:51.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:52.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:52.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:52.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:52.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:53.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:53.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:53.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:53.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:54.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:54.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:54.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:54.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:55.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:55.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:55.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:55.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:56.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:56.618+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:56.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:56.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:57.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:57.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:57.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:57.755+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:58.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:58.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:58.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:58.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:59.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:38:59.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:38:59.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:38:59.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:00.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:00.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:00.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:00.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:01.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:01.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:01.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:01.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:02.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:02.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:02.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:02.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:03.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:03.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:03.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:03.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:04.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:04.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:04.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:04.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:05.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:05.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:05.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:05.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:06.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:06.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:06.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:06.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:07.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:07.421+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:07.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:07.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:08.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:08.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:08.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:08.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:09.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:09.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:09.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:09.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:10.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:10.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:10.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:10.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:11.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:11.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:11.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:11.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:12.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:12.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:12.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:12.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:13.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:13.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:13.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:13.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:14.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:14.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:14.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:14.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:15.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:15.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:15.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:15.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:16.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:16.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:16.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:16.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:17.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:17.580+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:17.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:17.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:18.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:18.586+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:18.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:18.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:19.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:19.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:19.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:19.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:20.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:20.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:20.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:20.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:21.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:21.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:21.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:21.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:22.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:22.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:22.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:22.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:23.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:23.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:23.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:23.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:24.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:24.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:24.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:24.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:25.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:25.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:25.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:25.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:26.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:26.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:26.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:26.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:27.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:27.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:27.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:27.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:28.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:28.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:28.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:28.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:29.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:29.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:29.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:29.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:30.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:30.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:30.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:30.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:31.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:31.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:31.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:31.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:32.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:32.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:32.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:32.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:33.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:33.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:33.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:33.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:34.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:34.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:34.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:34.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:35.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:35.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:35.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:35.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:36.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:36.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:36.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:36.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:37.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:37.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:37.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:37.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:38.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:38.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:38.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:38.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:39.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:39.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:39.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:39.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:40.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:40.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:40.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:40.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:41.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:41.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:41.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:41.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:42.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:42.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:42.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:42.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:43.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:43.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:43.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:43.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:44.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:44.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:44.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:44.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:45.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:45.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:45.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:45.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:46.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:46.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:46.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:46.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:47.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:47.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:47.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:47.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:48.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:48.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:48.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:48.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:49.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:49.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:49.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:49.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:50.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:50.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:50.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:50.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:51.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:51.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:51.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:51.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:52.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:52.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:52.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:52.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:53.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:53.548+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:53.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:53.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:54.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:54.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:54.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:54.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:55.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:55.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:55.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:55.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:56.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:56.530+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:56.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:56.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:57.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:57.498+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:57.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:57.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:58.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:58.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:58.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:58.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:59.503 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:39:59.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:39:59.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:39:59.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:00.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:00.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:00.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:00.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:01.470 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:01.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:01.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:01.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:02.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:02.520+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:02.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:02.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:03.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:03.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:03.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:03.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:04.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:04.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:04.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:04.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:05.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:05.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:05.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:05.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:06.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:06.485+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:06.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:06.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:07.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:07.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:07.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:07.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:08.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:08.451+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:08.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:08.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:09.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:09.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:09.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:09.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:10.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:10.454+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:10.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:10.623+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:11.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:11.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:11.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:11.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:12.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:12.429+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:12.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:12.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:13.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:13.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:13.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:13.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:14.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:14.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:14.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:14.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:15.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:15.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:15.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:15.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:16.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:16.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:16.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:16.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:17.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:17.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:17.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:17.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:18.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:18.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:18.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:18.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:19.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:19.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:19.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:19.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:20.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:20.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:20.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:20.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:21.457 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:21.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:21.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:21.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:22.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:22.417+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:22.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:22.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:23.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:23.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:23.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:23.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:24.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:24.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:24.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:24.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:25.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:25.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:25.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:25.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:26.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:26.487+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:26.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:26.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:27.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:27.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:27.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:27.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:28.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:28.485+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:28.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:28.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:29.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:29.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:29.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:29.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:30.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:30.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:30.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:30.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:31.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:31.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:31.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:31.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:32.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:32.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:32.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:32.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:33.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:33.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:33.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:33.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:34.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:34.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:34.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:34.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:35.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:35.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:35.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:35.888+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:36.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:36.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:36.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:36.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:37.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:37.579+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:37.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:37.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:38.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:38.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:38.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:38.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:39.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:39.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:39.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:39.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:40.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:40.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:40.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:40.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:41.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:41.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:41.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:41.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:42.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:42.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:42.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:42.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:43.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:43.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:43.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:43.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:44.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:44.530+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:44.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:44.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:45.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:45.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:45.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:45.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:46.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:46.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:46.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:46.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:47.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:47.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:47.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:47.936+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:48.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:48.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:48.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:48.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:49.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:49.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:49.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:49.994+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:50.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:50.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:50.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:50.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:51.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:51.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:51.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:51.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:52.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:52.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:52.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:52.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:53.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:53.451+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:53.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:53.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:54.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:54.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:54.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:54.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:55.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:55.385+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:55.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:55.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:56.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:56.350+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:56.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:56.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:57.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:57.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:57.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:57.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:58.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:58.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:58.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:58.910+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:59.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:40:59.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:40:59.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:40:59.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:00.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:00.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:00.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:00.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:01.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:01.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:02.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:02.011+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:02.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:02.239+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:03.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:03.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:03.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:03.282+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:04.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:04.030+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:04.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:04.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:04.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:04.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:05.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:05.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:05.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:05.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:06.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:06.184+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:06.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:06.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:07.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:07.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:07.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:07.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:08.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:08.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:08.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:08.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:09.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:09.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:09.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:09.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:10.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:10.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:10.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:10.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:11.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:11.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:11.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:11.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:12.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:12.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:12.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:12.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:13.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:13.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:13.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:13.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:14.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:14.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:14.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:14.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:15.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:15.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:15.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:15.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:15.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:15.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:16.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:16.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:16.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:16.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:17.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:17.917+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:17.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:17.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:18.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:18.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:18.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:18.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:19.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:19.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:19.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:19.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:20.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:20.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:20.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:20.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:21.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:21.904+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:21.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:21.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:22.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:22.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:22.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:22.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:23.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:23.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:24.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:24.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:24.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:24.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:25.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:25.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:25.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:25.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:26.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:26.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:26.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:26.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:27.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:27.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:27.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:27.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:27.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:27.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:28.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:28.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:28.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:28.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:29.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:29.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:30.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:30.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:30.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:30.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:31.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:31.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:32.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:32.007+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:32.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:32.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:32.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:32.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:33.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:33.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:34.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:34.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:34.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:34.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:35.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:35.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:35.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:35.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:36.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:36.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:36.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:36.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:37.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:37.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:37.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:37.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:38.003 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:38.003+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:38.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:38.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:39.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:39.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:39.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:39.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:40.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:40.007+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:40.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:40.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:41.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:41.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:41.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:41.029+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:42.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:42.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:42.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:42.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:43.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:43.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:43.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:43.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:44.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:44.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:44.074 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:44.075+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:45.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:45.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:45.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:45.066+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:46.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:46.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:46.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:46.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:47.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:47.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:47.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:47.151+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:48.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:48.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:48.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:48.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:49.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:49.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:49.130 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:49.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:50.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:50.087+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:50.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:50.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:51.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:51.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:51.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:51.155+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:52.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:52.014+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:52.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:52.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:53.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:53.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:53.214 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:53.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:54.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:54.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:54.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:54.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:55.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:55.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:55.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:55.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:56.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:56.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:56.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:56.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:57.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:57.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:57.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:57.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:57.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:57.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:58.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:58.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:59.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:59.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:59.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:41:59.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:41:59.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:41:59.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:00.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:00.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:00.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:00.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:01.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:01.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:01.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:01.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:02.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:02.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:02.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:02.978+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:03.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:03.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:03.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:03.945+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:04.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:04.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:04.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:04.934+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:05.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:05.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:05.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:05.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:06.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:06.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:06.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:06.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:07.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:07.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:07.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:07.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:08.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:08.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:08.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:08.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:09.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:09.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:09.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:09.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:10.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:10.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:10.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:10.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:11.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:11.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:11.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:11.911+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:12.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:12.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:12.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:12.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:13.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:13.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:13.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:13.897+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:14.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:14.197+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:14.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:14.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:15.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:15.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:15.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:15.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:16.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:16.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:16.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:16.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:17.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:17.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:17.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:17.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:18.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:18.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:18.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:18.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:19.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:19.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:19.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:19.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:20.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:20.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:20.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:20.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:21.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:21.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:21.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:21.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:22.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:22.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:22.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:22.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:23.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:23.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:23.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:23.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:24.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:24.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:24.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:24.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:25.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:25.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:25.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:25.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:26.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:26.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:26.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:26.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:27.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:27.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:27.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:27.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:28.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:28.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:28.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:28.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:29.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:29.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:29.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:29.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:30.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:30.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:30.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:30.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:31.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:31.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:31.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:31.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:32.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:32.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:32.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:32.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:33.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:33.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:33.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:33.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:34.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:34.220+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:34.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:34.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:35.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:35.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:35.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:35.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:36.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:36.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:36.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:36.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:37.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:37.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:37.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:37.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:38.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:38.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:38.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:38.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:39.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:39.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:39.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:39.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:40.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:40.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:40.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:40.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:41.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:41.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:41.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:41.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:42.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:42.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:42.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:42.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:43.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:43.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:43.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:43.621+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:44.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:44.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:44.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:44.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:45.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:45.244+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:45.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:45.624+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:46.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:46.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:46.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:46.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:47.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:47.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:47.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:47.580+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:48.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:48.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:48.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:48.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:49.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:49.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:49.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:49.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:50.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:50.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:50.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:50.624+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:51.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:51.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:51.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:51.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:52.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:52.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:52.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:52.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:53.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:53.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:53.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:53.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:54.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:54.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:54.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:54.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:55.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:55.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:55.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:55.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:56.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:56.157+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:56.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:56.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:57.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:57.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:57.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:57.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:58.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:58.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:58.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:58.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:59.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:42:59.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:42:59.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:42:59.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:00.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:00.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:00.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:00.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:01.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:01.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:01.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:01.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:02.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:02.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:02.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:02.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:03.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:03.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:03.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:03.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:04.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:04.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:04.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:04.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:05.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:05.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:05.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:05.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:06.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:06.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:06.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:06.586+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:07.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:07.145+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:07.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:07.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:08.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:08.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:08.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:08.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:09.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:09.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:09.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:09.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:10.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:10.083+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:10.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:10.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:11.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:11.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:11.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:11.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:12.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:12.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:12.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:12.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:13.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:13.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:13.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:13.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:14.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:14.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:14.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:14.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:15.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:15.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:15.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:15.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:16.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:16.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:16.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:16.623+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:17.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:17.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:17.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:17.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:17.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:17.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:18.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:18.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:19.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:19.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:19.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:19.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:19.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:19.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:20.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:20.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:20.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:20.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:21.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:21.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:21.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:21.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:22.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:22.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:22.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:22.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:23.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:23.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:23.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:23.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:24.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:24.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:24.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:24.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:25.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:25.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:26.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:26.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:26.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:26.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:26.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:26.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:27.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:27.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:27.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:27.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:28.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:28.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:28.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:28.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:29.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:29.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:29.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:29.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:30.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:30.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:30.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:30.984+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:31.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:31.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:32.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:32.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:32.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:32.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:32.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:32.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:33.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:33.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:33.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:33.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:34.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:34.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:34.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:34.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:35.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:35.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:35.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:35.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:36.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:36.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:36.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:36.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:37.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:37.586+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:37.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:37.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:38.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:38.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:38.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:38.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:39.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:39.618+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:39.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:39.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:40.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:40.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:40.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:40.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:41.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:41.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:41.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:41.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:42.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:42.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:42.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:42.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:43.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:43.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:43.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:43.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:44.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:44.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:44.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:44.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:45.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:45.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:45.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:45.704+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:46.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:46.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:46.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:46.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:47.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:47.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:47.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:47.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:48.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:48.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:48.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:48.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:49.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:49.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:49.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:49.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:50.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:50.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:50.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:50.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:51.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:51.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:51.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:51.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:52.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:52.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:52.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:52.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:53.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:53.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:53.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:53.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:54.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:54.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:54.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:54.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:55.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:55.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:55.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:55.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:56.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:56.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:56.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:56.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:57.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:57.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:57.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:57.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:58.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:58.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:58.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:58.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:59.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:43:59.713+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:43:59.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:43:59.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:00.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:00.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:00.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:00.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:01.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:01.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:01.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:01.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:02.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:02.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:02.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:02.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:03.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:03.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:03.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:03.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:04.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:04.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:04.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:04.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:05.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:05.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:05.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:05.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:06.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:06.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:06.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:06.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:07.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:07.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:07.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:07.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:08.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:08.740+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:08.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:08.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:09.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:09.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:10.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:10.019+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:10.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:10.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:11.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:11.016+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:11.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:11.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:11.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:11.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:12.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:12.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:13.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:13.011+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:13.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:13.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:14.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:14.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:14.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:14.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:15.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:15.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:15.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:15.692+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:16.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:16.010+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:16.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:16.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:17.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:17.007+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:17.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:17.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:18.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:18.057+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:18.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:18.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:19.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:19.082+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:19.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:19.709+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:20.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:20.066+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:20.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:20.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:21.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:21.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:21.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:21.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:22.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:22.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:22.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:22.717+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:23.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:23.015+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:23.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:23.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:24.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:24.019+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:24.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:24.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:25.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:25.010+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:25.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:25.698+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:26.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:26.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:26.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:26.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:27.041 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:27.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:27.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:27.745+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:28.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:28.029+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:28.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:28.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:29.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:29.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:29.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:29.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:30.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:30.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:30.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:30.739+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:31.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:31.134+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:31.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:31.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:32.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:32.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:32.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:32.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:33.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:33.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:33.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:33.765+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:34.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:34.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:34.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:34.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:35.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:35.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:35.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:35.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:36.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:36.166+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:36.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:36.768+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:37.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:37.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:37.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:37.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:38.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:38.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:38.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:38.739+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:39.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:39.136+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:39.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:39.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:40.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:40.154+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:40.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:40.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:41.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:41.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:41.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:41.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:42.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:42.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:42.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:42.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:43.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:43.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:43.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:43.745+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:44.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:44.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:44.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:44.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:45.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:45.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:45.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:45.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:46.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:46.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:46.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:46.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:47.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:47.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:47.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:47.762+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:48.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:48.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:48.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:48.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:49.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:49.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:49.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:49.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:50.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:50.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:50.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:50.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:51.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:51.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:51.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:51.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:52.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:52.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:52.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:52.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:53.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:53.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:53.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:53.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:54.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:54.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:54.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:54.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:55.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:55.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:55.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:55.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:56.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:56.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:56.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:56.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:57.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:57.189+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:57.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:57.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:58.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:58.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:58.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:58.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:59.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:44:59.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:44:59.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:44:59.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:00.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:00.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:00.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:00.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:01.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:01.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:01.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:01.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:02.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:02.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:02.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:02.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:03.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:03.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:03.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:03.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:04.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:04.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:04.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:04.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:05.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:05.178+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:05.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:05.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:06.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:06.208+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:06.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:06.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:07.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:07.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:07.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:07.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:08.221 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T19:45:08.221+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffb6c034140 msgr2=0x7ffb6c047380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T19:45:08.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:08.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:08.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:08.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:09.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:09.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:09.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:09.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:10.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:10.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:10.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:10.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:11.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:11.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:11.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:11.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:12.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:12.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:12.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:12.460+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:13.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:13.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:13.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:13.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:14.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:14.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:14.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:14.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:15.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:15.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:15.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:15.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:16.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:16.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:16.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:16.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:17.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:17.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:17.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:17.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:18.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:18.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:18.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:18.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:19.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:19.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:19.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:19.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:20.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:20.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:20.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:20.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:21.227 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:21.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:21.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:21.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:22.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:22.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:22.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:22.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:23.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:23.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:23.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:23.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:24.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:24.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:24.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:24.595+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:25.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:25.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:25.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:25.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:26.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:26.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:26.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:26.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:27.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:27.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:27.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:27.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:28.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:28.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:28.688 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:28.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:29.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:29.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:29.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:29.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:30.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:30.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:30.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:30.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:31.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:31.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:31.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:31.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:32.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:32.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:32.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:32.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:33.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:33.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:33.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:33.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:34.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:34.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:34.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:34.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:35.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:35.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:35.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:35.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:36.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:36.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:36.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:36.596+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:37.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:37.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:37.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:37.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:38.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:38.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:38.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:38.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:39.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:39.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:39.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:39.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:40.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:40.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:40.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:40.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:41.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:41.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:41.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:41.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:42.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:42.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:42.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:42.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:43.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:43.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:43.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:43.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:44.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:44.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:44.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:44.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:45.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:45.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:45.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:45.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:46.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:46.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:46.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:46.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:47.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:47.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:47.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:47.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:48.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:48.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:48.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:48.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:49.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:49.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:49.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:49.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:50.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:50.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:50.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:50.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:51.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:51.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:51.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:51.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:52.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:52.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:52.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:52.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:53.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:53.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:53.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:53.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:54.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:54.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:54.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:54.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:55.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:55.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:55.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:55.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:56.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:56.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:56.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:56.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:57.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:57.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:57.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:57.692+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:58.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:58.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:58.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:58.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:59.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:45:59.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:45:59.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:45:59.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:00.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:00.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:00.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:00.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:01.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:01.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:01.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:01.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:02.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:02.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:02.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:02.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:03.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:03.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:03.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:03.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:04.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:04.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:04.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:04.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:05.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:05.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:05.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:05.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:06.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:06.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:06.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:06.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:07.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:07.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:07.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:07.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:08.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:08.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:08.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:08.720+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:09.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:09.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:09.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:09.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:10.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:10.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:10.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:10.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:11.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:11.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:11.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:11.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:12.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:12.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:12.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:12.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:13.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:13.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:13.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:13.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:14.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:14.415+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:14.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:14.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:15.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:15.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:15.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:15.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:16.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:16.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:16.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:16.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:17.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:17.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:17.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:17.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:18.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:18.395+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:18.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:18.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:19.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:19.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:19.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:19.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:20.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:20.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:20.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:20.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:21.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:21.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:21.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:21.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:22.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:22.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:22.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:22.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:23.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:23.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:23.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:23.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:24.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:24.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:24.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:24.575+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:25.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:25.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:25.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:25.606+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:26.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:26.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:26.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:26.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:27.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:27.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:27.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:27.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:28.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:28.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:28.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:28.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:29.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:29.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:29.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:29.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:30.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:30.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:30.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:30.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:31.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:31.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:31.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:31.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:32.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:32.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:32.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:32.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:33.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:33.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:33.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:33.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:34.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:34.327+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:34.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:34.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:35.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:35.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:35.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:35.596+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:36.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:36.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:36.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:36.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:37.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:37.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:37.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:37.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:38.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:38.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:38.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:38.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:39.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:39.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:39.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:39.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:40.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:40.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:40.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:40.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:41.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:41.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:41.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:41.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:42.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:42.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:42.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:42.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:43.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:43.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:43.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:43.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:44.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:44.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:44.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:44.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:45.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:45.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:45.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:45.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:46.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:46.359+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:46.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:46.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:47.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:47.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:47.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:47.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:48.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:48.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:48.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:48.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:49.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:49.357+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:49.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:49.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:50.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:50.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:50.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:50.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:51.364 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:51.364+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:51.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:51.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:52.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:52.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:52.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:52.607+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:53.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:53.286+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:53.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:53.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:54.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:54.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:54.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:54.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:55.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:55.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:55.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:55.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:56.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:56.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:56.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:56.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:57.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:57.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:57.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:57.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:58.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:58.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:58.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:58.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:59.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:46:59.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:46:59.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:46:59.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:00.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:00.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:00.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:00.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:01.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:01.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:01.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:01.539+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:02.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:02.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:02.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:02.498+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:03.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:03.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:03.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:03.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:04.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:04.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:04.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:04.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:05.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:05.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:05.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:05.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:06.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:06.228+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:06.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:06.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:07.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:07.278+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:07.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:07.641+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:08.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:08.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:08.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:08.607+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:09.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:09.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:09.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:09.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:10.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:10.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:10.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:10.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:11.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:11.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:11.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:11.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:12.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:12.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:12.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:12.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:13.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:13.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:13.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:13.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:14.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:14.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:14.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:14.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:15.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:15.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:15.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:15.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:16.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:16.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:16.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:16.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:17.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:17.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:17.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:17.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:18.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:18.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:18.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:18.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:19.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:19.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:19.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:19.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:20.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:20.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:20.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:20.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:21.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:21.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:21.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:21.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:22.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:22.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:22.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:22.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:23.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:23.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:23.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:23.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:24.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:24.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:24.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:24.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:25.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:25.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:25.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:25.568+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:26.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:26.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:26.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:26.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:27.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:27.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:27.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:27.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:28.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:28.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:28.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:28.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:29.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:29.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:29.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:29.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:30.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:30.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:30.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:30.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:31.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:31.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:31.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:31.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:32.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:32.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:32.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:32.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:33.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:33.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:33.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:33.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:34.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:34.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:34.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:34.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:35.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:35.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:35.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:35.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:36.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:36.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:36.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:36.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:37.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:37.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:37.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:37.607+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:38.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:38.305+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:38.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:38.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:39.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:39.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:39.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:39.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:40.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:40.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:40.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:40.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:41.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:41.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:41.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:41.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:42.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:42.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:42.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:42.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:43.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:43.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:43.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:43.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:44.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:44.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:44.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:44.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:45.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:45.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:45.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:45.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:46.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:46.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:46.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:46.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:47.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:47.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:47.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:47.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:48.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:48.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:48.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:48.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:49.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:49.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:49.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:49.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:50.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:50.277+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:50.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:50.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:51.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:51.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:51.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:51.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:52.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:52.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:52.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:52.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:53.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:53.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:53.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:53.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:54.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:54.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:54.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:54.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:55.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:55.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:55.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:55.718+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:56.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:56.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:56.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:56.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:57.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:57.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:57.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:57.687+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:58.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:58.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:58.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:58.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:59.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:47:59.258+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:47:59.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:47:59.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:00.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:00.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:00.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:00.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:01.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:01.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:01.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:01.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:02.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:02.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:02.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:02.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:03.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:03.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:03.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:03.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:04.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:04.301+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:04.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:04.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:05.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:05.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:05.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:05.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:06.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:06.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:06.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:06.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:07.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:07.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:07.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:07.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:08.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:08.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:08.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:08.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:09.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:09.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:09.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:09.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:10.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:10.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:10.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:10.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:11.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:11.286+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:11.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:11.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:12.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:12.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:12.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:12.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:13.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:13.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:13.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:13.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:14.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:14.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:14.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:14.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:15.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:15.372+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:15.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:15.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:16.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:16.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:16.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:16.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:17.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:17.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:17.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:17.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:18.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:18.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:18.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:18.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:19.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:19.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:19.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:19.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:20.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:20.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:20.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:20.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:21.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:21.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:21.503 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:21.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:22.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:22.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:22.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:22.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:23.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:23.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:23.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:23.521+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:24.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:24.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:24.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:24.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:25.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:25.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:25.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:25.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:26.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:26.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:26.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:26.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:27.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:27.372+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:27.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:27.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:28.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:28.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:28.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:28.484+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:29.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:29.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:29.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:29.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:30.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:30.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:30.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:30.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:31.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:31.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:31.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:31.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:32.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:32.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:32.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:32.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:33.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:33.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:33.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:33.562+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:34.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:34.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:34.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:34.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:35.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:35.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:35.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:35.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:36.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:36.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:36.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:36.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:37.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:37.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:37.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:37.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:38.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:38.416+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:38.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:38.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:39.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:39.390+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:39.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:39.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:40.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:40.375+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:40.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:40.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:41.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:41.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:41.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:41.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:42.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:42.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:42.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:42.444+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:43.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:43.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:43.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:43.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:44.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:44.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:44.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:44.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:45.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:45.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:45.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:45.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:46.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:46.435+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:46.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:46.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:47.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:47.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:47.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:47.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:48.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:48.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:48.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:48.493+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:49.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:49.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:49.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:49.460+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:50.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:50.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:50.451 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:50.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:51.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:51.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:51.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:51.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:52.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:52.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:52.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:52.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:53.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:53.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:53.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:53.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:54.427 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:54.427+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:54.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:54.449+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:55.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:55.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:55.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:55.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:56.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:56.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:56.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:56.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:57.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:57.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:57.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:57.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:58.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:58.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:58.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:58.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:59.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:48:59.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:48:59.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:48:59.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:00.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:00.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:00.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:00.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:01.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:01.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:01.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:01.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:02.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:02.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:02.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:02.490+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:03.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:03.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:03.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:03.521+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:04.490 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:04.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:04.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:04.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:05.490 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:05.490+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:05.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:05.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:06.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:06.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:06.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:06.545+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:07.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:07.548+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:07.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:07.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:08.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:08.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:08.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:08.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:09.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:09.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:09.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:09.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:10.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:10.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:10.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:10.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:11.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:11.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:11.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:11.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:12.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:12.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:12.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:12.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:13.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:13.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:13.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:13.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:14.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:14.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:14.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:14.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:15.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:15.487+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:15.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:15.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:16.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:16.481+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:16.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:16.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:17.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:17.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:17.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:17.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:18.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:18.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:18.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:18.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:19.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:19.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:19.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:19.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:20.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:20.508+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:20.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:20.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:21.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:21.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:21.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:21.481+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:22.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:22.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:22.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:22.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:23.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:23.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:23.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:23.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:24.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:24.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:24.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:24.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:25.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:25.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:25.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:25.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:26.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:26.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:26.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:26.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:27.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:27.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:27.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:27.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:28.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:28.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:28.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:28.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:29.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:29.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:29.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:29.478+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:30.369 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:30.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:30.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:30.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:31.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:31.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:31.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:31.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:32.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:32.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:32.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:32.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:33.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:33.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:33.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:33.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:34.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:34.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:34.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:34.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:35.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:35.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:35.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:35.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:36.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:36.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:36.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:36.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:37.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:37.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:37.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:37.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:38.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:38.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:38.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:38.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:39.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:39.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:39.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:39.566+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:40.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:40.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:40.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:40.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:41.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:41.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:41.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:41.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:42.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:42.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:42.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:42.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:43.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:43.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:43.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:43.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:44.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:44.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:44.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:44.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:45.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:45.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:45.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:45.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:46.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:46.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:46.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:46.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:47.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:47.182+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:47.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:47.490+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:48.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:48.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:48.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:48.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:49.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:49.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:49.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:49.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:50.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:50.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:50.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:50.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:51.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:51.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:51.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:51.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:52.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:52.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:52.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:52.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:53.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:53.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:53.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:53.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:54.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:54.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:54.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:54.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:55.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:55.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:55.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:55.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:56.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:56.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:56.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:56.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:57.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:57.083+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:57.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:57.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:58.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:58.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:58.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:58.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:59.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:49:59.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:49:59.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:49:59.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:00.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:00.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:00.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:00.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:01.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:01.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:01.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:01.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:02.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:02.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:02.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:02.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:03.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:03.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:03.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:03.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:04.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:04.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:04.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:04.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:05.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:05.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:05.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:05.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:06.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:06.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:06.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:06.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:07.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:07.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:07.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:07.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:08.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:08.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:08.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:08.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:09.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:09.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:09.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:09.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:10.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:10.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:10.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:10.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:11.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:11.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:11.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:11.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:12.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:12.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:12.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:12.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:13.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:13.199+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:13.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:13.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:14.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:14.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:14.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:14.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:15.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:15.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:15.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:15.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:16.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:16.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:16.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:16.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:17.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:17.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:17.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:17.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:18.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:18.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:18.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:18.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:19.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:19.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:19.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:19.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:20.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:20.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:20.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:20.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:21.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:21.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:21.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:21.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:22.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:22.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:22.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:22.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:23.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:23.139+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:23.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:23.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:24.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:24.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:24.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:24.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:25.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:25.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:25.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:25.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:26.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:26.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:26.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:26.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:27.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:27.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:27.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:27.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:28.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:28.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:28.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:28.876+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:29.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:29.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:29.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:29.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:30.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:30.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:30.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:30.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:31.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:31.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:31.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:31.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:32.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:32.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:32.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:32.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:33.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:33.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:33.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:33.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:34.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:34.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:34.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:34.793+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:35.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:35.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:35.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:35.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:36.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:36.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:36.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:36.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:37.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:37.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:37.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:37.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:38.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:38.349+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:38.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:38.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:39.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:39.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:39.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:39.856+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:40.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:40.379+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:40.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:40.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:41.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:41.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:41.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:41.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:42.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:42.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:42.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:42.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:43.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:43.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:43.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:43.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:44.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:44.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:44.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:44.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:45.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:45.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:45.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:45.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:46.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:46.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:46.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:46.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:47.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:47.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:47.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:47.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:48.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:48.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:48.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:48.867+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:49.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:49.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:49.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:49.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:50.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:50.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:50.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:50.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:51.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:51.412+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:51.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:51.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:52.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:52.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:52.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:52.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:53.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:53.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:53.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:53.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:54.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:54.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:54.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:54.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:55.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:55.400+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:55.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:55.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:56.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:56.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:56.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:56.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:57.378 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:57.378+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:57.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:57.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:58.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:58.356+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:58.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:58.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:59.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:50:59.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:50:59.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:50:59.823+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:00.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:00.417+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:00.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:00.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:01.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:01.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:01.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:01.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:02.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:02.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:02.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:02.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:03.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:03.423+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:03.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:03.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:04.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:04.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:04.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:04.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:05.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:05.460+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:05.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:05.823+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:06.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:06.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:06.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:06.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:07.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:07.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:07.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:07.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:08.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:08.443+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:08.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:08.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:09.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:09.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:09.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:09.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:10.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:10.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:10.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:10.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:11.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:11.332+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:11.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:11.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:12.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:12.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:12.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:12.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:13.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:13.299+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:13.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:13.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:14.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:14.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:14.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:14.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:15.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:15.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:15.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:15.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:16.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:16.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:16.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:16.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:17.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:17.361+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:17.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:17.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:18.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:18.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:18.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:18.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:19.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:19.390+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:19.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:19.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:20.408 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:20.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:20.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:20.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:21.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:21.422+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:21.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:21.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:22.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:22.410+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:22.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:22.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:23.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:23.419+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:23.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:23.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:24.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:24.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:24.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:24.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:25.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:25.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:25.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:25.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:26.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:26.405+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:26.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:26.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:27.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:27.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:27.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:27.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:28.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:28.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:28.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:28.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:29.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:29.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:29.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:29.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:30.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:30.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:30.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:30.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:31.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:31.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:31.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:31.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:32.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:32.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:32.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:32.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:33.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:33.606+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:33.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:33.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:34.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:34.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:34.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:34.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:35.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:35.577+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:35.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:35.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:36.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:36.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:36.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:36.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:37.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:37.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:37.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:37.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:38.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:38.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:38.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:38.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:39.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:39.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:39.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:39.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:40.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:40.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:40.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:40.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:41.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:41.506+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:41.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:41.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:42.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:42.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:42.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:42.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:43.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:43.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:43.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:43.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:44.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:44.534+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:44.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:44.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:45.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:45.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:45.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:45.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:46.501 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:46.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:46.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:46.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:47.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:47.504+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:47.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:47.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:48.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:48.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:48.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:48.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:49.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:49.421+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:49.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:49.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:50.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:50.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:50.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:50.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:51.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:51.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:51.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:51.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:52.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:52.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:52.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:52.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:53.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:53.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:53.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:53.651+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:54.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:54.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:54.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:54.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:55.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:55.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:55.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:55.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:56.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:56.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:56.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:56.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:57.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:57.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:57.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:57.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:58.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:58.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:58.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:58.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:59.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:51:59.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:51:59.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:51:59.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:00.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:00.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:00.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:00.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:01.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:01.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:01.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:01.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:02.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:02.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:02.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:02.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:03.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:03.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:03.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:03.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:04.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:04.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:04.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:04.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:05.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:05.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:05.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:05.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:06.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:06.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:06.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:06.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:07.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:07.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:07.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:07.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:08.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:08.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:08.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:08.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:09.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:09.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:09.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:09.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:10.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:10.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:10.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:10.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:11.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:11.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:11.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:11.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:12.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:12.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:12.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:12.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:13.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:13.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:13.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:13.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:14.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:14.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:14.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:14.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:15.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:15.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:15.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:15.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:16.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:16.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:16.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:16.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:17.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:17.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:17.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:17.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:18.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:18.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:18.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:18.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:19.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:19.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:19.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:19.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:20.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:20.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:20.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:20.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:21.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:21.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:21.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:21.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:22.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:22.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:22.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:22.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:23.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:23.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:23.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:23.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:24.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:24.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:24.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:24.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:25.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:25.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:25.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:25.680+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:26.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:26.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:26.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:26.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:27.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:27.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:27.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:27.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:28.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:28.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:28.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:28.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:29.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:29.605+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:29.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:29.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:30.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:30.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:30.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:30.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:31.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:31.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:31.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:31.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:32.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:32.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:32.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:32.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:33.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:33.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:33.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:33.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:34.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:34.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:34.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:34.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:35.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:35.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:35.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:35.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:36.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:36.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:36.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:36.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:37.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:37.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:37.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:37.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:38.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:38.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:38.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:38.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:39.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:39.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:39.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:39.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:40.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:40.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:40.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:40.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:41.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:41.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:41.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:41.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:42.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:42.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:42.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:42.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:43.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:43.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:43.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:43.856+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:44.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:44.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:44.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:44.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:45.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:45.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:45.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:45.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:46.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:46.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:46.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:46.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:47.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:47.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:47.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:47.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:48.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:48.704+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:48.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:48.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:49.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:49.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:49.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:49.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:50.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:50.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:50.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:50.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:51.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:51.769+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:51.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:51.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:52.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:52.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:52.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:52.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:53.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:53.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:53.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:53.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:54.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:54.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:54.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:54.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:55.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:55.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:55.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:55.947+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:56.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:56.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:56.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:56.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:57.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:57.764+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:57.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:57.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:58.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:58.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:58.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:58.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:59.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:52:59.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:52:59.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:52:59.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:00.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:00.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:00.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:00.987+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:01.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:01.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:01.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:01.947+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:02.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:02.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:02.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:02.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:03.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:03.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:03.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:03.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:04.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:04.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:04.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:04.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:05.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:05.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:05.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:05.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:06.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:06.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:06.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:06.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:07.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:07.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:07.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:07.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:08.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:08.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:08.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:08.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:09.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:09.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:10.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:10.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:10.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:10.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:11.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:11.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:11.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:11.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:12.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:12.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:12.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:12.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:13.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:13.049+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:13.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:13.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:14.053 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:14.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:14.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:14.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:15.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:15.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:15.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:15.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:16.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:16.094+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:16.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:16.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:17.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:17.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:17.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:17.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:18.125 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:18.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:18.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:18.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:19.085 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:19.085+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:19.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:19.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:20.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:20.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:20.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:20.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:21.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:21.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:21.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:21.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:22.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:22.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:22.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:22.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:23.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:23.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:23.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:23.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:24.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:24.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:24.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:24.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:25.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:25.257+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:25.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:25.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:26.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:26.267+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:26.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:26.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:27.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:27.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:27.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:27.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:28.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:28.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:28.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:28.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:29.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:29.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:29.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:29.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:30.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:30.277+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:30.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:30.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:31.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:31.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:31.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:31.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:32.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:32.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:32.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:32.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:33.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:33.277+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:33.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:33.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:34.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:34.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:34.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:34.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:35.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:35.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:35.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:35.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:36.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:36.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:36.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:36.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:37.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:37.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:37.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:37.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:38.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:38.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:38.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:38.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:39.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:39.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:39.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:39.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:40.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:40.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:40.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:40.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:41.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:41.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:41.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:41.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:42.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:42.409+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:42.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:42.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:43.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:43.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:43.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:43.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:44.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:44.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:44.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:44.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:45.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:45.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:45.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:45.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:46.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:46.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:46.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:46.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:47.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:47.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:47.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:47.803+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:48.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:48.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:48.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:48.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:49.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:49.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:49.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:49.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:50.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:50.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:50.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:50.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:51.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:51.376+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:51.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:51.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:52.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:52.359+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:52.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:52.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:53.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:53.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:53.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:53.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:54.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:54.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:54.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:54.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:55.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:55.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:55.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:55.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:56.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:56.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:56.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:56.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:57.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:57.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:57.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:57.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:58.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:58.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:58.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:58.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:59.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:53:59.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:53:59.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:53:59.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:00.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:00.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:00.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:00.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:01.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:01.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:01.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:01.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:02.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:02.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:02.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:02.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:03.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:03.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:03.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:03.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:04.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:04.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:04.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:04.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:05.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:05.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:05.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:05.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:06.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:06.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:06.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:06.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:07.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:07.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:07.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:07.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:08.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:08.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:08.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:08.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:09.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:09.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:09.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:09.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:10.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:10.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:10.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:10.764+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:11.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:11.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:11.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:11.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:12.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:12.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:12.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:12.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:13.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:13.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:13.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:13.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:14.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:14.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:14.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:14.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:15.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:15.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:15.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:15.830+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:16.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:16.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:16.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:16.798+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:17.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:17.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:17.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:17.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:18.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:18.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:18.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:18.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:19.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:19.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:19.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:19.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:20.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:20.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:20.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:20.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:21.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:21.343+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:21.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:21.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:22.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:22.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:22.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:22.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:23.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:23.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:23.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:23.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:24.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:24.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:24.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:24.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:25.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:25.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:26.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:26.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:26.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:26.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:26.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:26.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:27.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:27.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:27.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:27.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:28.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:28.354+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:28.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:28.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:29.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:29.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:29.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:29.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:30.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:30.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:30.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:30.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:31.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:31.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:31.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:31.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:32.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:32.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:32.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:32.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:33.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:33.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:33.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:33.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:34.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:34.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:34.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:34.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:35.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:35.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:35.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:35.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:36.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:36.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:36.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:36.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:37.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:37.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:37.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:37.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:38.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:38.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:38.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:38.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:39.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:39.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:39.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:39.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:40.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:40.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:40.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:40.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:41.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:41.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:41.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:41.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:42.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:42.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:43.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:43.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:43.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:43.191+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:44.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:44.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:44.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:44.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:45.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:45.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:45.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:45.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:46.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:46.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:46.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:46.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:47.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:47.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:47.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:47.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:48.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:48.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:48.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:48.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:48.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:48.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:49.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:49.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:49.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:49.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:50.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:50.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:51.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:51.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:51.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:51.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:52.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:52.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:52.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:52.258+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:53.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:53.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:53.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:53.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:54.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:54.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:54.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:54.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:55.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:55.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:55.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:55.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:56.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:56.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:56.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:56.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:57.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:57.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:57.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:57.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:58.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:58.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:58.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:58.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:59.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:54:59.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:54:59.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:54:59.284+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:00.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:00.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:00.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:00.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:01.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:01.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:01.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:01.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:02.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:02.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:02.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:02.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:03.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:03.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:03.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:03.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:04.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:04.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:04.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:04.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:05.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:05.349+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:05.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:05.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:06.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:06.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:06.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:06.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:07.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:07.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:07.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:07.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:08.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:08.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:08.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:08.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:09.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:09.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:09.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:09.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:10.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:10.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:10.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:10.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:11.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:11.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:11.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:11.449+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:12.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:12.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:12.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:12.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:13.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:13.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:13.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:13.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:14.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:14.246+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:14.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:14.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:15.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:15.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:15.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:15.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:16.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:16.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:16.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:16.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:17.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:17.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:17.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:17.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:18.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:18.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:18.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:18.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:19.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:19.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:19.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:19.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:20.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:20.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:20.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:20.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:21.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:21.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:21.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:21.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:22.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:22.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:22.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:22.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:23.317 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:23.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:23.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:23.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:24.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:24.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:24.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:24.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:25.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:25.261+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:25.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:25.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:26.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:26.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:26.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:26.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:27.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:27.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:27.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:27.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:28.302 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:28.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:28.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:28.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:29.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:29.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:29.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:29.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:30.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:30.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:30.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:30.428+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:31.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:31.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:31.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:31.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:32.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:32.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:32.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:32.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:33.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:33.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:33.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:33.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:34.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:34.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:34.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:34.407+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:35.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:35.272+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:35.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:35.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:36.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:36.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:36.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:36.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:37.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:37.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:37.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:37.323+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:38.317 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:38.318+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:38.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:38.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:39.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:39.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:39.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:39.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:40.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:40.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:40.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:40.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:41.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:41.318+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:41.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:41.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:42.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:42.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:42.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:42.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:43.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:43.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:43.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:43.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:44.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:44.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:44.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:44.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:45.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:45.288+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:45.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:45.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:46.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:46.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:46.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:46.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:47.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:47.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:47.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:47.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:48.351 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:48.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:48.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:48.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:49.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:49.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:49.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:49.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:50.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:50.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:50.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:50.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:51.403 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:51.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:51.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:51.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:52.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:52.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:52.408 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:52.409+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:53.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:53.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:53.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:53.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:54.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:54.380+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:54.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:54.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:55.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:55.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:55.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:55.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:56.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:56.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:56.427 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:56.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:57.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:57.421+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:57.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:57.442+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:58.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:58.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:58.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:58.417+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:59.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:55:59.375+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:55:59.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:55:59.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:00.378 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:00.377+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:00.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:00.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:01.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:01.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:01.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:01.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:02.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:02.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:02.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:02.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:03.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:03.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:03.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:03.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:04.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:04.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:04.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:04.390+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:05.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:05.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:05.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:05.385+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:06.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:06.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:06.355 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:06.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:07.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:07.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:07.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:07.375+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:08.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:08.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:08.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:08.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:09.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:09.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:09.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:09.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:10.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:10.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:10.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:10.332+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:11.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:11.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:11.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:11.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:12.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:12.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:12.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:12.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:13.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:13.263+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:13.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:13.349+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:14.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:14.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:14.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:14.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:15.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:15.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:15.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:15.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:16.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:16.270+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:16.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:16.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:17.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:17.239+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:17.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:17.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:18.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:18.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:18.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:18.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:19.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:19.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:19.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:19.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:20.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:20.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:20.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:20.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:21.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:21.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:21.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:21.323+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:22.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:22.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:22.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:22.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:23.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:23.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:23.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:23.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:24.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:24.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:24.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:24.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:25.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:25.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:25.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:25.354+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:26.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:26.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:26.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:26.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:27.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:27.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:27.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:27.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:28.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:28.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:28.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:28.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:29.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:29.139+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:29.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:29.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:30.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:30.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:30.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:30.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:31.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:31.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:31.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:31.390+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:32.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:32.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:32.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:32.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:33.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:33.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:33.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:33.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:34.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:34.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:34.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:34.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:35.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:35.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:35.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:35.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:36.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:36.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:36.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:36.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:37.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:37.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:37.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:37.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:38.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:38.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:38.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:38.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:39.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:39.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:39.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:39.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:40.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:40.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:40.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:40.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:41.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:41.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:41.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:41.327+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:42.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:42.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:42.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:42.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:43.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:43.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:43.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:43.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:44.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:44.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:44.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:44.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:45.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:45.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:45.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:45.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:46.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:46.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:46.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:46.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:47.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:47.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:47.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:47.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:48.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:48.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:48.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:48.301+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:49.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:49.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:49.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:49.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:50.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:50.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:50.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:50.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:51.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:51.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:51.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:51.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:52.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:52.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:52.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:52.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:53.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:53.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:53.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:53.274+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:54.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:54.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:54.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:54.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:55.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:55.197+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:55.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:55.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:56.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:56.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:56.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:56.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:57.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:57.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:57.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:57.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:58.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:58.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:58.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:58.257+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:59.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:56:59.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:56:59.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:56:59.228+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:00.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:00.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:00.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:00.267+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:01.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:01.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:01.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:01.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:02.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:02.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:02.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:02.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:03.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:03.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:03.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:03.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:04.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:04.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:04.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:04.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:05.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:05.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:05.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:05.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:06.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:06.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:06.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:06.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:07.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:07.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:07.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:07.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:08.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:08.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:08.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:08.277+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:09.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:09.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:09.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:09.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:10.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:10.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:10.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:10.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:11.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:11.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:11.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:11.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:12.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:12.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:12.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:12.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:13.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:13.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:13.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:13.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:14.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:14.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:14.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:14.361+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:15.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:15.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:15.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:15.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:16.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:16.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:16.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:16.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:17.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:17.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:17.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:17.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:18.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:18.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:18.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:18.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:19.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:19.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:19.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:19.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:20.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:20.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:20.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:20.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:21.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:21.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:21.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:21.342+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:22.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:22.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:22.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:22.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:23.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:23.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:23.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:23.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:24.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:24.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:24.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:24.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:25.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:25.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:25.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:25.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:26.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:26.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:26.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:26.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:27.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:27.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:27.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:27.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:28.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:28.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:28.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:28.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:29.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:29.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:29.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:29.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:30.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:30.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:30.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:30.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:31.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:31.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:31.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:31.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:32.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:32.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:32.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:32.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:33.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:33.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:33.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:33.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:34.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:34.244+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:34.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:34.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:35.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:35.278+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:35.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:35.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:36.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:36.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:36.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:36.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:37.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:37.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:37.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:37.298+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:38.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:38.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:38.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:38.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:39.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:39.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:39.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:39.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:40.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:40.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:40.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:40.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:41.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:41.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:41.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:41.367+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:42.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:42.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:42.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:42.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:43.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:43.218+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:43.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:43.381+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:44.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:44.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:44.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:44.407+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:45.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:45.187+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:45.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:45.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:46.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:46.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:46.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:46.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:47.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:47.111+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:47.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:47.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:48.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:48.106+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:48.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:48.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:49.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:49.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:49.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:49.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:50.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:50.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:50.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:50.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:51.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:51.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:51.467 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:51.467+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:52.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:52.178+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:52.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:52.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:53.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:53.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:53.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:53.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:54.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:54.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:54.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:54.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:55.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:55.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:55.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:55.421+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:56.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:56.202+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:56.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:56.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:57.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:57.189+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:57.459 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:57.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:58.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:58.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:58.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:58.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:59.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:57:59.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:57:59.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:57:59.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:00.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:00.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:00.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:00.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:01.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:01.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:01.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:01.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:02.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:02.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:02.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:02.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:03.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:03.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:03.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:03.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:04.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:04.181+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:04.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:04.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:05.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:05.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:05.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:05.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:06.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:06.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:06.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:06.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:07.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:07.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:07.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:07.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:08.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:08.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:08.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:08.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:09.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:09.339+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:09.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:09.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:10.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:10.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:10.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:10.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:11.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:11.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:11.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:11.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:12.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:12.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:12.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:12.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:13.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:13.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:13.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:13.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:14.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:14.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:14.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:14.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:15.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:15.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:15.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:15.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:16.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:16.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:16.459 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:16.458+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:17.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:17.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:17.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:17.467+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:18.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:18.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:18.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:18.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:19.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:19.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:19.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:19.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:20.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:20.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:20.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:20.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:21.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:21.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:21.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:21.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:22.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:22.380+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:22.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:22.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:23.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:23.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:23.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:23.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:24.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:24.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:24.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:24.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:25.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:25.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:25.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:25.508+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:26.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:26.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:26.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:26.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:27.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:27.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:27.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:27.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:28.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:28.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:28.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:28.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:29.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:29.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:29.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:29.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:30.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:30.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:30.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:30.579+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:31.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:31.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:31.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:31.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:32.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:32.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:32.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:32.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:33.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:33.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:33.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:33.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:34.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:34.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:34.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:34.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:35.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:35.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:35.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:35.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:36.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:36.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:36.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:36.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:37.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:37.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:37.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:37.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:38.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:38.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:38.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:38.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:39.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:39.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:39.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:39.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:40.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:40.318+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:40.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:40.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:41.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:41.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:41.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:41.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:42.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:42.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:42.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:42.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:43.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:43.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:43.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:43.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:44.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:44.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:44.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:44.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:45.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:45.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:45.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:45.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:46.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:46.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:46.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:46.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:47.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:47.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:47.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:47.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:48.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:48.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:48.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:48.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:49.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:49.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:49.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:49.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:50.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:50.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:50.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:50.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:51.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:51.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:51.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:51.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:52.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:52.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:52.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:52.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:53.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:53.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:53.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:53.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:54.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:54.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:54.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:54.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:55.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:55.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:55.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:55.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:56.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:56.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:56.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:56.756+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:57.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:57.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:57.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:57.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:58.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:58.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:58.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:58.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:59.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:58:59.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:58:59.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:58:59.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:00.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:00.274+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:00.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:00.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:01.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:01.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:01.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:01.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:02.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:02.327+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:02.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:02.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:03.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:03.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:03.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:03.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:04.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:04.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:04.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:04.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:05.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:05.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:05.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:05.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:06.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:06.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:06.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:06.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:07.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:07.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:07.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:07.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:08.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:08.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:08.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:08.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:09.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:09.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:09.764 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:09.765+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:10.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:10.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:10.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:10.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:11.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:11.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:11.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:11.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:12.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:12.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:12.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:12.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:13.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:13.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:13.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:13.739+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:14.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:14.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:14.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:14.765+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:15.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:15.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:15.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:15.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:16.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:16.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:16.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:16.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:17.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:17.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:17.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:17.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:18.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:18.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:18.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:18.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:19.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:19.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:19.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:19.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:20.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:20.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:20.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:20.758+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:21.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:21.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:21.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:21.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:22.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:22.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:22.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:22.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:23.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:23.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:23.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:23.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:24.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:24.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:24.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:24.792+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:25.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:25.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:25.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:25.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:26.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:26.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:26.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:26.769+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:27.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:27.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:27.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:27.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:28.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:28.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:28.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:28.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:29.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:29.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:29.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:29.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:30.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:30.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:30.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:30.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:31.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:31.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:31.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:31.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:32.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:32.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:32.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:32.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:33.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:33.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:33.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:33.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:34.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:34.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:34.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:34.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:35.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:35.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:35.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:35.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:36.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:36.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:36.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:36.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:37.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:37.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:37.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:37.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:38.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:38.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:38.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:38.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:39.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:39.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:39.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:39.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:40.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:40.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:40.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:40.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:41.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:41.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:41.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:41.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:42.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:42.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:42.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:42.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:43.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:43.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:43.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:43.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:44.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:44.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:44.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:44.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:45.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:45.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:45.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:45.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:46.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:46.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:46.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:46.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:47.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:47.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:47.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:47.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:48.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:48.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:48.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:48.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:49.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:49.602+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:49.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:49.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:50.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:50.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:50.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:50.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:51.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:51.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:51.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:51.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:52.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:52.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:52.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:52.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:53.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:53.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:53.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:53.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:54.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:54.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:54.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:54.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:55.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:55.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:55.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:55.800+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:56.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:56.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:56.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:56.799+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:57.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:57.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:57.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:57.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:58.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:58.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:58.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:58.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:59.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T19:59:59.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T19:59:59.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T19:59:59.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:00.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:00.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:00.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:00.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:01.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:01.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:01.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:01.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:02.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:02.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:02.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:02.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:03.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:03.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:03.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:03.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:04.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:04.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:04.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:04.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:05.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:05.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:05.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:05.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:06.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:06.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:06.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:06.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:07.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:07.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:07.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:07.786+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:08.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:08.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:08.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:08.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:09.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:09.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:09.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:09.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:10.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:10.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:10.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:10.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:11.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:11.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:11.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:11.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:12.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:12.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:12.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:12.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:13.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:13.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:13.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:13.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:14.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:14.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:14.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:14.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:15.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:15.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:15.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:15.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:16.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:16.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:16.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:16.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:17.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:17.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:17.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:17.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:18.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:18.624+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:18.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:18.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:19.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:19.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:19.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:19.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:20.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:20.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:20.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:20.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:21.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:21.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:21.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:21.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:22.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:22.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:22.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:22.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:23.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:23.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:23.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:23.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:24.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:24.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:24.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:24.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:25.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:25.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:25.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:25.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:26.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:26.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:26.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:26.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:27.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:27.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:27.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:27.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:28.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:28.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:28.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:28.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:29.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:29.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:29.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:29.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:30.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:30.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:30.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:30.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:31.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:31.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:31.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:31.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:32.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:32.896+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:32.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:32.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:33.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:33.910+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:33.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:33.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:34.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:34.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:34.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:34.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:35.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:35.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:35.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:35.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:36.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:36.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:36.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:36.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:37.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:37.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:37.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:37.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:38.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:38.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:38.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:38.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:39.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:39.896+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:40.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:40.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:40.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:40.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:40.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:40.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:41.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:41.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:41.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:41.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:42.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:42.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:42.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:42.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:43.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:43.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:43.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:43.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:44.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:44.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:45.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:45.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:45.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:45.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:45.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:45.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:46.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:46.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:46.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:46.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:47.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:47.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:47.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:47.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:48.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:48.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:48.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:48.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:49.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:49.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:49.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:49.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:50.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:50.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:51.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:51.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:51.844 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:51.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:52.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:52.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:52.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:52.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:53.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:53.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:53.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:53.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:53.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:53.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:54.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:54.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:54.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:54.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:55.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:55.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:55.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:55.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:56.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:56.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:56.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:56.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:57.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:57.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:57.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:57.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:58.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:58.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:58.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:58.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:59.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:00:59.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:00:59.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:00:59.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:00.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:00.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:00.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:00.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:01.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:01.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:01.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:01.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:02.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:02.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:02.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:02.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:03.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:03.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:03.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:03.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:04.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:04.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:04.863 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:04.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:05.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:05.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:05.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:05.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:06.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:06.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:06.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:06.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:07.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:07.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:07.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:07.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:08.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:08.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:08.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:08.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:09.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:09.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:09.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:09.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:10.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:10.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:10.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:10.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:11.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:11.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:11.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:11.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:12.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:12.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:12.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:12.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:13.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:13.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:13.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:13.933+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:14.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:14.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:14.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:14.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:15.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:15.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:16.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:16.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:16.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:16.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:17.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:17.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:17.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:17.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:18.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:18.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:18.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:18.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:19.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:19.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:19.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:19.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:19.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:19.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:20.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:20.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:21.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:21.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:21.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:21.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:22.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:22.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:22.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:22.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:23.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:23.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:23.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:23.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:24.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:24.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:24.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:24.672+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:25.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:25.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:25.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:25.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:26.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:26.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:26.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:26.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:27.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:27.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:27.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:27.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:28.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:28.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:28.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:28.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:29.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:29.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:29.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:29.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:30.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:30.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:30.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:30.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:31.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:31.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:31.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:31.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:32.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:32.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:32.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:32.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:33.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:33.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:33.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:33.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:34.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:34.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:34.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:34.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:35.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:35.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:35.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:35.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:35.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:35.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:36.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:36.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:36.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:36.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:37.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:37.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:37.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:37.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:38.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:38.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:38.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:38.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:39.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:39.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:39.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:39.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:40.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:40.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:41.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:41.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:41.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:41.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:42.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:42.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:42.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:42.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:43.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:43.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:43.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:43.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:44.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:44.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:44.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:44.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:45.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:45.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:45.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:45.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:46.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:46.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:46.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:46.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:47.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:47.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:47.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:47.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:48.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:48.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:48.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:48.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:49.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:49.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:49.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:49.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:50.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:50.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:50.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:50.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:51.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:51.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:51.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:51.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:52.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:52.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:52.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:52.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:53.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:53.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:53.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:53.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:54.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:54.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:54.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:54.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:55.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:55.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:55.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:55.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:56.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:56.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:56.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:56.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:57.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:57.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:57.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:57.651+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:58.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:58.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:58.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:58.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:59.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:01:59.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:01:59.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:01:59.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:00.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:00.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:00.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:00.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:01.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:01.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:01.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:01.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:02.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:02.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:02.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:02.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:03.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:03.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:03.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:03.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:04.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:04.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:04.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:04.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:04.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:04.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:05.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:05.569+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:05.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:05.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:06.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:06.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:07.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:07.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:07.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:07.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:08.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:08.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:08.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:08.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:09.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:09.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:09.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:09.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:10.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:10.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:10.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:10.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:11.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:11.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:11.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:11.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:12.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:12.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:12.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:12.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:13.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:13.001+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:13.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:13.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:14.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:14.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:14.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:14.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:14.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:14.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:15.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:15.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:15.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:15.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:16.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:16.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:16.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:16.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:17.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:17.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:18.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:18.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:18.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:18.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:19.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:19.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:19.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:19.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:19.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:19.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:20.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:20.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:21.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:21.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:21.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:21.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:21.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:21.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:22.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:22.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:22.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:22.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:23.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:23.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:23.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:23.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:24.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:24.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:24.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:24.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:25.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:25.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:25.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:25.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:26.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:26.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:26.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:26.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:27.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:27.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:27.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:27.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:28.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:28.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:28.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:28.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:29.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:29.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:29.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:29.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:30.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:30.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:30.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:30.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:31.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:31.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:31.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:31.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:32.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:32.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:32.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:32.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:33.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:33.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:33.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:33.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:34.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:34.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:35.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:35.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:35.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:35.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:35.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:35.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:36.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:36.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:37.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:37.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:37.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:37.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:38.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:38.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:38.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:38.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:39.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:39.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:39.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:39.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:40.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:40.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:40.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:40.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:41.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:41.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:41.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:41.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:42.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:42.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:42.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:42.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:42.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:42.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:43.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:43.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:43.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:43.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:44.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:44.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:44.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:44.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:45.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:45.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:45.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:45.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:46.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:46.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:46.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:46.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:47.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:47.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:47.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:47.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:48.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:48.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:48.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:48.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:49.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:49.545+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:49.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:49.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:50.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:50.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:50.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:50.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:51.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:51.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:51.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:51.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:52.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:52.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:52.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:52.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:53.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:53.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:53.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:53.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:54.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:54.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:54.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:54.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:55.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:55.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:56.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:56.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:56.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:56.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:57.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:57.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:57.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:57.586+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:58.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:58.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:58.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:58.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:59.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:02:59.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:02:59.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:02:59.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:00.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:00.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:00.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:00.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:01.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:01.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:01.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:01.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:02.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:02.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:02.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:02.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:03.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:03.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:03.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:03.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:04.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:04.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:04.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:04.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:05.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:05.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:05.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:05.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:06.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:06.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:06.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:06.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:07.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:07.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:07.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:07.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:08.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:08.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:08.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:08.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:09.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:09.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:09.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:09.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:10.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:10.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:10.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:10.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:11.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:11.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:11.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:11.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:12.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:12.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:12.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:12.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:13.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:13.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:13.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:13.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:14.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:14.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:14.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:14.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:15.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:15.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:15.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:15.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:16.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:16.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:16.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:16.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:17.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:17.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:17.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:17.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:18.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:18.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:18.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:18.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:19.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:19.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:19.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:19.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:20.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:20.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:20.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:20.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:21.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:21.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:21.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:21.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:22.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:22.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:22.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:22.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:23.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:23.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:23.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:23.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:24.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:24.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:24.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:24.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:24.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:24.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:25.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:25.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:26.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:26.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:26.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:26.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:26.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:26.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:27.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:27.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:27.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:27.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:28.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:28.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:28.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:28.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:29.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:29.657+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:29.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:29.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:30.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:30.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:30.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:30.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:31.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:31.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:31.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:31.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:32.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:32.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:32.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:32.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:33.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:33.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:33.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:33.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:34.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:34.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:34.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:34.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:35.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:35.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:36.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:36.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:36.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:36.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:37.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:37.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:37.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:37.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:38.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:38.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:38.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:38.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:39.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:39.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:39.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:39.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:40.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:40.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:40.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:40.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:41.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:41.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:41.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:41.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:42.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:42.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:42.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:42.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:43.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:43.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:43.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:43.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:44.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:44.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:44.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:44.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:45.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:45.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:45.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:45.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:46.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:46.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:46.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:46.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:47.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:47.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:47.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:47.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:47.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:47.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:48.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:48.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:48.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:48.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:49.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:49.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:50.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:50.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:50.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:50.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:51.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:51.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:51.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:51.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:52.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:52.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:52.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:52.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:53.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:53.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:53.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:53.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:54.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:54.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:54.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:54.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:55.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:55.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:55.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:55.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:56.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:56.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:56.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:56.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:57.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:57.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:57.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:57.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:58.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:58.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:58.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:58.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:59.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:03:59.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:03:59.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:03:59.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:00.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:00.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:00.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:00.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:01.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:01.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:01.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:01.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:02.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:02.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:02.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:02.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:03.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:03.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:03.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:03.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:04.259 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:04.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:04.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:04.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:05.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:05.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:05.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:05.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:06.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:06.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:06.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:06.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:07.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:07.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:07.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:07.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:08.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:08.182+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:08.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:08.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:09.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:09.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:09.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:09.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:10.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:10.220+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:10.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:10.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:11.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:11.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:11.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:11.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:12.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:12.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:12.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:12.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:13.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:13.233+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:13.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:13.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:14.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:14.220+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:14.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:14.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:15.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:15.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:15.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:15.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:16.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:16.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:16.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:16.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:17.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:17.272+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:17.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:17.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:18.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:18.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:18.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:18.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:19.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:19.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:19.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:19.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:20.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:20.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:20.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:20.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:21.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:21.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:21.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:21.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:22.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:22.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:22.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:22.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:23.356 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:23.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:23.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:23.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:24.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:24.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:24.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:24.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:25.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:25.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:25.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:25.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:26.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:26.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:26.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:26.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:27.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:27.394+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:27.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:27.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:28.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:28.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:28.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:28.751+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:29.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:29.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:29.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:29.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:30.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:30.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:30.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:30.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:31.369 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:31.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:31.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:31.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:32.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:32.384+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:32.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:32.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:33.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:33.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:33.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:33.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:34.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:34.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:34.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:34.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:35.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:35.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:35.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:35.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:36.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:36.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:36.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:36.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:37.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:37.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:37.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:37.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:38.317 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:38.316+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:38.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:38.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:39.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:39.299+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:39.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:39.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:40.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:40.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:40.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:40.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:41.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:41.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:41.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:41.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:42.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:42.248+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:42.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:42.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:43.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:43.269+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:43.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:43.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:44.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:44.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:44.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:44.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:45.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:45.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:45.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:45.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:46.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:46.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:46.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:46.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:47.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:47.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:47.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:47.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:48.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:48.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:48.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:48.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:49.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:49.419+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:49.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:49.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:50.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:50.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:50.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:50.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:51.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:51.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:51.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:51.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:52.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:52.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:52.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:52.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:53.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:53.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:53.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:53.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:54.457 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:54.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:54.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:54.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:55.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:55.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:55.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:55.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:56.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:56.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:56.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:56.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:57.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:57.424+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:57.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:57.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:58.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:58.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:58.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:58.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:59.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:04:59.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:04:59.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:04:59.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:00.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:00.490+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:00.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:00.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:01.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:01.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:01.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:01.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:02.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:02.443+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:02.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:02.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:03.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:03.485+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:03.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:03.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:04.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:04.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:04.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:04.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:05.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:05.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:05.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:05.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:06.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:06.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:06.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:06.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:07.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:07.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:07.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:07.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:08.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:08.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:08.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:08.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:09.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:09.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:09.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:09.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:10.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:10.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:10.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:10.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:11.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:11.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:11.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:11.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:12.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:12.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:12.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:12.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:13.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:13.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:13.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:13.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:14.436 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:14.436+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:14.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:14.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:15.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:15.454+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:15.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:15.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:16.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:16.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:16.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:16.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:17.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:17.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:17.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:17.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:18.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:18.419+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:18.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:18.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:19.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:19.397+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:19.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:19.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:20.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:20.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:20.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:20.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:21.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:21.449+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:21.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:21.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:22.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:22.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:22.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:22.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:23.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:23.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:23.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:23.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:24.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:24.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:24.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:24.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:25.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:25.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:25.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:25.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:26.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:26.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:26.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:26.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:27.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:27.385+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:27.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:27.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:28.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:28.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:28.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:28.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:29.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:29.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:29.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:29.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:30.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:30.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:30.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:30.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:31.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:31.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:31.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:31.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:32.479 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:32.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:32.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:32.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:33.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:33.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:33.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:33.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:34.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:34.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:35.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:35.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:35.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:35.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:35.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:35.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:36.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:36.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:36.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:36.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:37.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:37.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:37.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:37.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:38.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:38.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:38.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:38.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:39.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:39.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:39.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:39.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:40.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:40.562+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:40.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:40.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:41.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:41.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:41.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:41.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:42.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:42.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:42.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:42.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:43.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:43.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:43.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:43.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:44.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:44.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:44.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:44.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:45.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:45.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:45.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:45.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:46.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:46.508+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:46.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:46.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:47.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:47.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:47.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:47.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:48.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:48.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:48.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:48.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:49.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:49.548+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:49.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:49.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:50.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:50.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:50.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:50.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:51.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:51.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:51.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:51.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:52.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:52.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:52.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:52.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:53.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:53.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:53.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:53.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:54.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:54.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:54.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:54.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:55.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:55.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:55.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:55.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:56.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:56.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:56.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:56.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:57.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:57.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:57.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:57.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:58.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:58.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:58.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:58.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:59.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:05:59.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:05:59.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:05:59.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:00.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:00.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:00.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:00.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:01.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:01.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:01.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:01.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:02.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:02.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:02.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:02.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:03.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:03.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:03.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:03.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:04.599 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:04.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:04.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:04.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:05.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:05.641+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:05.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:05.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:06.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:06.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:06.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:06.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:07.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:07.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:07.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:07.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:08.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:08.577+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:08.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:08.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:09.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:09.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:09.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:09.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:10.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:10.542+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:10.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:10.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:11.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:11.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:11.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:11.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:12.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:12.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:12.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:12.856+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:13.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:13.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:13.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:13.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:14.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:14.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:14.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:14.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:15.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:15.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:15.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:15.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:16.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:16.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:16.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:16.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:17.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:17.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:17.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:17.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:18.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:18.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:18.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:18.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:19.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:19.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:19.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:19.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:20.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:20.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:20.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:20.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:21.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:21.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:21.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:21.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:22.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:22.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:22.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:22.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:23.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:23.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:23.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:23.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:24.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:24.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:24.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:24.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:25.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:25.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:25.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:25.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:26.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:26.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:26.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:26.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:27.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:27.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:27.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:27.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:28.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:28.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:28.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:28.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:29.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:29.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:29.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:29.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:30.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:30.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:30.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:30.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:31.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:31.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:31.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:31.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:32.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:32.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:32.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:32.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:33.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:33.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:33.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:33.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:34.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:34.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:34.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:34.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:35.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:35.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:35.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:35.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:36.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:36.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:36.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:36.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:37.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:37.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:37.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:37.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:38.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:38.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:38.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:38.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:39.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:39.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:39.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:39.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:40.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:40.692+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:40.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:40.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:41.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:41.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:41.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:41.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:42.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:42.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:42.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:42.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:43.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:43.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:43.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:43.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:44.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:44.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:44.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:44.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:45.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:45.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:45.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:45.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:46.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:46.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:46.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:46.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:47.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:47.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:47.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:47.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:48.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:48.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:48.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:48.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:49.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:49.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:49.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:49.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:50.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:50.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:50.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:50.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:51.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:51.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:51.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:51.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:52.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:52.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:52.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:52.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:53.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:53.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:53.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:53.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:54.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:54.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:54.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:54.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:55.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:55.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:55.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:55.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:56.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:56.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:56.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:56.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:57.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:57.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:57.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:57.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:58.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:58.687+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:58.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:58.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:59.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:06:59.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:06:59.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:06:59.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:00.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:00.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:00.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:00.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:01.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:01.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:01.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:01.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:02.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:02.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:02.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:02.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:03.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:03.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:03.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:03.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:04.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:04.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:04.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:04.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:05.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:05.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:05.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:05.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:06.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:06.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:06.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:06.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:07.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:07.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:07.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:07.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:08.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:08.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:08.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:08.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:09.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:09.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:09.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:09.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:10.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:10.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:10.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:10.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:11.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:11.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:11.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:11.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:12.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:12.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:12.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:12.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:13.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:13.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:13.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:13.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:14.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:14.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:14.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:14.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:15.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:15.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:15.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:15.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:16.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:16.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:16.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:16.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:17.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:17.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:17.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:17.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:18.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:18.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:18.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:18.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:19.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:19.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:20.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:20.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:20.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:20.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:20.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:20.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:21.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:21.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:21.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:21.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:22.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:22.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:22.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:22.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:23.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:23.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:23.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:23.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:24.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:24.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:24.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:24.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:25.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:25.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:25.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:25.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:26.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:26.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:26.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:26.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:27.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:27.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:27.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:27.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:28.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:28.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:28.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:28.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:29.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:29.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:29.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:29.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:30.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:30.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:30.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:30.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:31.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:31.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:31.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:31.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:32.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:32.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:32.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:32.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:33.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:33.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:33.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:33.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:34.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:34.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:34.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:34.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:35.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:35.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:35.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:35.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:36.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:36.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:36.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:36.895+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:37.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:37.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:37.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:37.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:38.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:38.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:38.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:38.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:39.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:39.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:39.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:39.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:40.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:40.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:40.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:40.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:41.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:41.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:41.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:41.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:42.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:42.713+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:42.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:42.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:43.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:43.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:43.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:43.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:44.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:44.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:44.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:44.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:45.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:45.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:45.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:45.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:46.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:46.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:46.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:46.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:47.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:47.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:47.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:47.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:48.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:48.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:48.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:48.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:49.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:49.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:49.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:49.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:50.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:50.756+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:50.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:50.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:51.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:51.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:51.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:51.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:52.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:52.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:52.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:52.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:53.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:53.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:53.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:53.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:54.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:54.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:55.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:55.013+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:55.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:55.695+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:56.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:56.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:56.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:56.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:56.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:56.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:57.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:57.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:58.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:58.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:58.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:58.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:58.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:07:58.983+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:07:59.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:07:59.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:00.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:00.003+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:00.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:00.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:01.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:01.050+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:01.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:01.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:02.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:02.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:02.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:02.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:03.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:03.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:03.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:03.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:04.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:04.050+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:04.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:04.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:05.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:05.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:05.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:05.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:06.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:06.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:06.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:06.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:07.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:07.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:07.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:07.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:08.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:08.133+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:08.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:08.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:09.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:09.097+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:09.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:09.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:10.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:10.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:10.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:10.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:11.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:11.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:11.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:11.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:12.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:12.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:12.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:12.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:13.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:13.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:13.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:13.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:14.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:14.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:14.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:14.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:15.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:15.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:15.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:15.567+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:16.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:16.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:16.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:16.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:17.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:17.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:17.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:17.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:18.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:18.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:18.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:18.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:19.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:19.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:19.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:19.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:20.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:20.114+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:20.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:20.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:21.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:21.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:21.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:21.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:22.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:22.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:22.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:22.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:23.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:23.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:23.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:23.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:24.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:24.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:24.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:24.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:25.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:25.189+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:25.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:25.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:26.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:26.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:26.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:26.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:27.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:27.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:27.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:27.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:28.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:28.210+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:28.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:28.798+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:29.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:29.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:29.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:29.844+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:30.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:30.205+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:30.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:30.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:31.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:31.222+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:31.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:31.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:32.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:32.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:32.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:32.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:33.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:33.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:33.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:33.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:34.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:34.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:34.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:34.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:35.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:35.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:35.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:35.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:36.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:36.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:36.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:36.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:37.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:37.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:37.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:37.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:38.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:38.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:38.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:38.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:39.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:39.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:39.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:39.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:40.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:40.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:40.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:40.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:41.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:41.301+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:41.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:41.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:42.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:42.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:42.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:42.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:43.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:43.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:43.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:43.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:44.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:44.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:44.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:44.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:45.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:45.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:45.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:45.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:46.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:46.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:46.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:46.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:47.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:47.305+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:47.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:47.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:48.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:48.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:48.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:48.749+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:49.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:49.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:49.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:49.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:50.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:50.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:50.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:50.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:51.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:51.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:51.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:51.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:52.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:52.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:52.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:52.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:53.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:53.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:53.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:53.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:54.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:54.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:54.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:54.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:55.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:55.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:55.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:55.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:56.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:56.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:56.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:56.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:57.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:57.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:57.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:57.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:58.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:58.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:58.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:58.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:59.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:08:59.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:08:59.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:08:59.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:00.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:00.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:00.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:00.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:01.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:01.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:01.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:01.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:02.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:02.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:02.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:02.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:03.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:03.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:03.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:03.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:04.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:04.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:04.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:04.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:05.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:05.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:05.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:05.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:06.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:06.357+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:06.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:06.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:07.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:07.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:07.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:07.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:08.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:08.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:08.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:08.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:09.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:09.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:09.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:09.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:10.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:10.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:10.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:10.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:11.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:11.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:11.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:11.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:12.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:12.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:12.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:12.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:13.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:13.364+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:13.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:13.596+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:14.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:14.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:14.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:14.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:15.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:15.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:15.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:15.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:16.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:16.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:16.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:16.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:17.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:17.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:17.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:17.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:18.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:18.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:18.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:18.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:19.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:19.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:19.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:19.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:20.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:20.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:20.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:20.678+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:21.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:21.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:21.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:21.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:22.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:22.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:22.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:22.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:23.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:23.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:23.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:23.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:24.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:24.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:24.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:24.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:25.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:25.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:25.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:25.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:26.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:26.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:26.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:26.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:27.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:27.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:27.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:27.720+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:28.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:28.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:28.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:28.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:29.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:29.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:29.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:29.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:30.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:30.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:30.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:30.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:31.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:31.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:31.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:31.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:32.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:32.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:32.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:32.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:33.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:33.160+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:33.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:33.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:34.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:34.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:34.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:34.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:35.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:35.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:35.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:35.755+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:36.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:36.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:36.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:36.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:37.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:37.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:37.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:37.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:38.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:38.206+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:38.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:38.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:39.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:39.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:39.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:39.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:40.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:40.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:40.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:40.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:41.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:41.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:41.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:41.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:42.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:42.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:42.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:42.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:43.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:43.181+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:43.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:43.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:44.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:44.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:44.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:44.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:45.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:45.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:45.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:45.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:46.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:46.122+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:46.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:46.792+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:47.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:47.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:47.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:47.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:48.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:48.134+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:48.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:48.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:49.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:49.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:49.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:49.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:50.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:50.062+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:50.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:50.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:51.061 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:51.060+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:51.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:51.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:52.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:52.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:52.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:52.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:53.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:53.047+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:53.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:53.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:54.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:54.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:54.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:54.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:54.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:54.987+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:55.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:55.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:55.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:55.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:56.742 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:56.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:56.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:56.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:57.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:57.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:57.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:57.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:58.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:58.678+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:58.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:58.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:59.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:09:59.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:09:59.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:09:59.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:00.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:00.698+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:00.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:00.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:01.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:01.717+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:01.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:01.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:02.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:02.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:02.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:02.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:03.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:03.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:03.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:03.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:04.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:04.709+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:04.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:04.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:05.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:05.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:05.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:05.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:06.688 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:06.687+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:06.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:06.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:07.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:07.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:08.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:08.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:08.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:08.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:09.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:09.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:09.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:09.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:10.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:10.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:10.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:10.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:11.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:11.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:11.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:11.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:12.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:12.099+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:12.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:12.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:13.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:13.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:13.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:13.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:14.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:14.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:14.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:14.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:15.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:15.128+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:15.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:15.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:16.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:16.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:16.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:16.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:17.152 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:17.151+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:17.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:17.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:18.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:18.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:18.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:18.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:19.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:19.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:19.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:19.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:20.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:20.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:20.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:20.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:21.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:21.149+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:21.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:21.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:22.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:22.179+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:22.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:22.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:23.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:23.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:23.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:23.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:24.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:24.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:24.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:24.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:25.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:25.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:25.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:25.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:26.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:26.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:26.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:26.709+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:27.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:27.179+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:27.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:27.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:28.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:28.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:28.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:28.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:29.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:29.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:29.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:29.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:30.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:30.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:30.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:30.740+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:31.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:31.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:31.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:31.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:32.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:32.208+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:32.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:32.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:33.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:33.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:33.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:33.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:34.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:34.191+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:34.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:34.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:35.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:35.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:35.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:35.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:36.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:36.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:36.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:36.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:37.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:37.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:37.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:37.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:38.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:38.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:38.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:38.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:39.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:39.134+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:39.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:39.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:40.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:40.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:40.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:40.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:41.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:41.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:41.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:41.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:42.091 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:42.090+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:42.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:42.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:43.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:43.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:43.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:43.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:44.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:44.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:44.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:44.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:45.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:45.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:45.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:45.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:46.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:46.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:46.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:46.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:47.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:47.094+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:47.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:47.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:48.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:48.073+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:48.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:48.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:49.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:49.102+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:49.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:49.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:50.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:50.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:50.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:50.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:51.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:51.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:51.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:51.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:52.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:52.099+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:52.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:52.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:53.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:53.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:53.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:53.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:54.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:54.141+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:54.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:54.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:55.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:55.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:55.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:55.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:56.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:56.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:56.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:56.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:57.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:57.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:58.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:58.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:58.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:58.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:59.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:10:59.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:10:59.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:10:59.139+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:00.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:00.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:00.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:00.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:01.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:01.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:01.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:01.129+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:02.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:02.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:02.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:02.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:03.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:03.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:03.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:03.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:04.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:04.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:04.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:04.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:04.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:04.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:05.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:05.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:06.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:06.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:06.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:06.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:07.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:07.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:07.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:07.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:07.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:07.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:08.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:08.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:09.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:09.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:09.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:09.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:10.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:10.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:10.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:10.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:11.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:11.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:11.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:11.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:12.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:12.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:12.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:12.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:13.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:13.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:13.164 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:13.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:14.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:14.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:14.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:14.195+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:15.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:15.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:15.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:15.205+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:15.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:15.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:16.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:16.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:17.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:17.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:17.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:17.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:17.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:17.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:18.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:18.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:18.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:18.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:19.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:19.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:19.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:20.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:20.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:20.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:20.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:20.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:21.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:21.257+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:21.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:21.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:22.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:22.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:22.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:22.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:23.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:23.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:23.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:23.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:24.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:24.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:24.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:24.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:25.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:25.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:25.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:25.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:26.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:26.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:26.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:26.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:27.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:27.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:27.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:27.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:28.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:28.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:28.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:28.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:29.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:29.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:29.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:29.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:30.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:30.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:30.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:30.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:31.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:31.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:31.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:31.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:32.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:32.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:32.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:32.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:33.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:33.260+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:33.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:33.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:34.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:34.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:34.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:34.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:35.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:35.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:35.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:35.800+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:36.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:36.359+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:36.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:36.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:37.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:37.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:37.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:37.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:38.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:38.394+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:38.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:38.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:39.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:39.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:39.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:39.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:40.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:40.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:40.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:40.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:41.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:41.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:41.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:41.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:42.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:42.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:42.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:42.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:43.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:43.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:43.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:43.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:44.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:44.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:44.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:44.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:45.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:45.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:45.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:45.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:46.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:46.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:46.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:46.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:47.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:47.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:47.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:47.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:48.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:48.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:48.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:48.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:49.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:49.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:49.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:49.830+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:50.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:50.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:50.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:50.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:51.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:51.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:51.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:51.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:52.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:52.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:52.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:52.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:53.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:53.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:53.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:53.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:54.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:54.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:54.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:54.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:55.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:55.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:55.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:55.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:56.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:56.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:56.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:56.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:57.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:57.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:57.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:57.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:58.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:58.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:58.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:58.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:59.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:11:59.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:11:59.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:11:59.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:00.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:00.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:00.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:00.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:01.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:01.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:01.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:01.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:02.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:02.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:02.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:02.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:03.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:03.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:03.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:03.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:04.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:04.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:04.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:04.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:05.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:05.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:05.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:05.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:06.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:06.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:06.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:06.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:07.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:07.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:07.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:07.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:08.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:08.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:08.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:08.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:09.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:09.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:09.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:09.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:10.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:10.736+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:10.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:10.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:11.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:11.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:11.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:11.802+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:12.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:12.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:12.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:12.762+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:13.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:13.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:13.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:13.790+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:14.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:14.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:14.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:14.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:15.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:15.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:15.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:15.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:16.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:16.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:16.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:16.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:17.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:17.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:17.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:17.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:18.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:18.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:18.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:18.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:19.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:19.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:19.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:19.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:20.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:20.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:20.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:20.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:21.730 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:21.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:21.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:21.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:22.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:22.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:22.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:22.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:23.710 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:23.709+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:23.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:23.792+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:24.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:24.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:24.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:24.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:25.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:25.768+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:25.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:25.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:26.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:26.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:26.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:26.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:27.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:27.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:27.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:27.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:28.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:28.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:28.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:28.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:29.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:29.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:29.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:29.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:30.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:30.717+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:30.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:30.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:31.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:31.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:31.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:31.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:32.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:32.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:32.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:32.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:33.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:33.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:33.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:33.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:34.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:34.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:34.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:34.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:35.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:35.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:35.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:35.890+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:36.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:36.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:36.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:36.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:37.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:37.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:37.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:37.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:38.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:38.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:38.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:38.888+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:39.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:39.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:39.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:39.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:40.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:40.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:40.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:40.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:41.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:41.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:41.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:41.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:42.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:42.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:42.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:42.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:43.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:43.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:43.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:43.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:44.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:44.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:44.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:44.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:45.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:45.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:45.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:45.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:46.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:46.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:46.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:46.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:47.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:47.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:47.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:47.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:48.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:48.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:48.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:48.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:49.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:49.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:49.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:49.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:50.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:50.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:50.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:50.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:51.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:51.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:51.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:51.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:52.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:52.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:52.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:52.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:53.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:53.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:53.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:53.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:54.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:54.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:54.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:54.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:55.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:55.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:55.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:55.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:56.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:56.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:56.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:56.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:57.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:57.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:57.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:57.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:58.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:58.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:58.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:58.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:59.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:12:59.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:12:59.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:12:59.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:00.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:00.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:00.712 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:00.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:01.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:01.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:01.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:01.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:02.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:02.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:02.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:02.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:03.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:03.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:03.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:03.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:04.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:04.672+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:04.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:04.718+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:05.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:05.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:05.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:05.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:06.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:06.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:06.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:06.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:07.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:07.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:07.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:07.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:08.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:08.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:08.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:08.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:09.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:09.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:09.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:09.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:10.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:10.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:10.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:10.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:11.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:11.805+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:11.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:11.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:12.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:12.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:12.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:12.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:13.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:13.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:13.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:13.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:14.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:14.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:14.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:14.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:15.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:15.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:15.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:15.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:16.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:16.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:16.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:16.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:17.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:17.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:17.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:17.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:18.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:18.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:18.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:18.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:19.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:19.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:19.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:19.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:20.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:20.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:20.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:20.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:21.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:21.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:22.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:22.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:22.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:22.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:23.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:23.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:23.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:23.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:24.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:24.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:24.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:24.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:24.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:24.984+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:25.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:25.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:26.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:26.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:26.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:26.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:27.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:27.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:27.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:27.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:28.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:28.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:28.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:28.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:29.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:29.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:29.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:29.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:30.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:30.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:30.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:30.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:31.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:31.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:31.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:31.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:32.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:32.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:32.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:32.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:33.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:33.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:33.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:33.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:34.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:34.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:34.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:34.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:35.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:35.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:35.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:35.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:36.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:36.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:36.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:36.876+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:37.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:37.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:37.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:37.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:38.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:38.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:38.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:38.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:39.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:39.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:39.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:39.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:40.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:40.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:40.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:40.934+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:41.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:41.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:41.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:41.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:42.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:42.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:42.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:42.950+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:43.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:43.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:43.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:43.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:44.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:44.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:44.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:44.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:45.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:45.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:45.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:45.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:46.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:46.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:46.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:46.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:47.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:47.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:47.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:47.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:48.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:48.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:49.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:49.016+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:49.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:49.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:50.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:50.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:50.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:50.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:51.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:51.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:51.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:51.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:52.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:52.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:52.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:52.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:53.064 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:53.063+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:53.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:53.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:54.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:54.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:54.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:54.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:55.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:55.049+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:55.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:55.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:56.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:56.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:56.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:56.044+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:56.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:56.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:57.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:57.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:57.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:57.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:58.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:58.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:58.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:58.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:59.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:13:59.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:13:59.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:13:59.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:00.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:00.070+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:00.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:00.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:01.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:01.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:01.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:01.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:02.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:02.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:02.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:02.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:03.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:03.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:03.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:03.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:04.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:04.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:04.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:04.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:04.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:04.982+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:05.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:05.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:05.993 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:05.992+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:06.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:06.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:06.956 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:06.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:07.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:07.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:07.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:07.947+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:08.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:08.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:08.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:08.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:09.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:09.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:09.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:09.945+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:10.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:10.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:10.995 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:10.994+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:11.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:11.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:12.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:12.019+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:12.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:12.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:13.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:13.007+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:13.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:13.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:13.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:13.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:14.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:14.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:14.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:14.987+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:15.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:15.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:15.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:15.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:16.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:16.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:16.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:16.957+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:17.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:17.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:17.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:17.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:18.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:18.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:19.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:19.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:19.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:19.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:20.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:20.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:20.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:20.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:21.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:21.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:21.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:21.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:22.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:22.075+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:22.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:22.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:23.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:23.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:24.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:24.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:24.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:24.080+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:25.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:25.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:25.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:25.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:25.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:25.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:26.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:26.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:26.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:26.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:27.113 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:27.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:27.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:27.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:28.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:28.141+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:28.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:28.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:29.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:29.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:29.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:29.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:30.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:30.203+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:30.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:30.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:31.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:31.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:32.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:32.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:32.128 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:32.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:33.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:33.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:33.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:33.111+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:34.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:34.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:34.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:34.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:35.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:35.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:35.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:35.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:36.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:36.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:36.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:36.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:36.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:36.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:37.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:37.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:37.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:37.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:38.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:38.123+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:38.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:38.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:39.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:39.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:39.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:39.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:40.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:40.108+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:40.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:40.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:41.074 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:41.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:41.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:41.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:42.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:42.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:42.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:42.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:43.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:43.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:43.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:43.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:44.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:44.068+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:44.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:44.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:45.053 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:45.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:45.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:45.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:46.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:46.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:46.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:46.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:46.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:46.978+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:47.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:47.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:47.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:47.996+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:48.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:48.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:49.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:49.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:49.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:49.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:50.063 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:50.062+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:50.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:50.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:51.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:51.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:51.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:51.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:52.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:52.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:52.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:52.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:53.041 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:53.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:53.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:53.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:54.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:54.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:54.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:54.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:55.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:55.028+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:55.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:55.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:56.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:56.060+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:56.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:56.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:57.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:57.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:57.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:57.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:58.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:58.073+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:58.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:58.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:59.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:14:59.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:14:59.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:14:59.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:00.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:00.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:00.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:00.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:01.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:01.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:01.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:01.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:02.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:02.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:02.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:02.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:03.136 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:03.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:03.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:03.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:04.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:04.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:04.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:04.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:05.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:05.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:05.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:05.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:06.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:06.101+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:06.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:06.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:07.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:07.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:07.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:07.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:08.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:08.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:08.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:08.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:09.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:09.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:09.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:09.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:10.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:10.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:10.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:10.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:11.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:11.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:11.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:11.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:12.102 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:12.102+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:12.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:12.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:13.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:13.059+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:13.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:13.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:14.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:14.054+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:14.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:14.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:15.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:15.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:15.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:15.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:16.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:16.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:16.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:16.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:17.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:17.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:17.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:17.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:18.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:18.054+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:18.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:18.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:19.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:19.052+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:19.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:19.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:20.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:20.081+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:20.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:20.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:21.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:21.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:21.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:21.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:22.074 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:22.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:22.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:22.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:23.095 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:23.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:23.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:23.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:24.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:24.114+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:25.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:25.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:25.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:25.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:26.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:26.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:26.123 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:26.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:26.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:26.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:27.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:27.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:28.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:28.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:28.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:28.198+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:28.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:28.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:29.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:29.160+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:29.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:29.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:30.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:30.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:30.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:30.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:31.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:31.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:31.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:31.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:32.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:32.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:33.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:33.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:33.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:33.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:34.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:34.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:34.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:34.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:35.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:35.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:35.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:35.144+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:36.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:36.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:36.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:36.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:37.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:37.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:37.121 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:37.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:38.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:38.101+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:38.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:38.166+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:39.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:39.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:39.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:39.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:40.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:40.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:40.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:40.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:41.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:41.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:41.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:41.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:42.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:42.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:42.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:42.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:43.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:43.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:43.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:43.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:44.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:44.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:44.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:44.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:45.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:45.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:45.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:45.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:46.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:46.239+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:46.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:46.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:47.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:47.244+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:47.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:47.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:48.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:48.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:48.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:48.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:49.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:49.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:49.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:49.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:50.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:50.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:50.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:50.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:51.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:51.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:51.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:51.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:52.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:52.215+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:52.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:52.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:53.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:53.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:53.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:53.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:54.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:54.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:54.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:54.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:55.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:55.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:55.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:55.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:56.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:56.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:56.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:56.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:57.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:57.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:57.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:57.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:58.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:58.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:58.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:58.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:59.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:15:59.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:15:59.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:15:59.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:00.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:00.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:00.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:00.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:01.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:01.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:01.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:01.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:02.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:02.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:02.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:02.327+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:03.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:03.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:03.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:03.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:04.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:04.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:04.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:04.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:05.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:05.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:05.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:05.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:06.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:06.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:06.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:06.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:07.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:07.278+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:07.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:07.316+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:08.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:08.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:08.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:08.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:09.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:09.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:09.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:09.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:10.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:10.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:10.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:10.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:11.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:11.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:11.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:11.356+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:12.284 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:12.283+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:12.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:12.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:13.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:13.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:13.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:13.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:14.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:14.359+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:14.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:14.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:15.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:15.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:15.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:15.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:16.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:16.354+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:16.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:16.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:17.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:17.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:17.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:17.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:18.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:18.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:18.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:18.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:19.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:19.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:19.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:19.412+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:20.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:20.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:20.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:20.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:21.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:21.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:21.459 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:21.458+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:22.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:22.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:22.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:22.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:23.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:23.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:23.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:23.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:24.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:24.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:24.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:24.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:25.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:25.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:25.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:25.443+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:26.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:26.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:26.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:26.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:27.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:27.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:27.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:27.499+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:28.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:28.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:28.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:28.506+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:29.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:29.323+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:29.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:29.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:30.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:30.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:30.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:30.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:31.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:31.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:31.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:31.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:32.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:32.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:32.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:32.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:33.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:33.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:33.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:33.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:34.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:34.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:34.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:34.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:35.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:35.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:35.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:35.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:36.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:36.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:36.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:36.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:37.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:37.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:37.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:37.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:38.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:38.299+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:38.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:38.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:39.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:39.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:39.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:39.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:40.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:40.286+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:40.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:40.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:41.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:41.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:41.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:41.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:42.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:42.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:42.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:42.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:43.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:43.210+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:43.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:43.389+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:44.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:44.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:44.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:44.365+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:45.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:45.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:45.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:45.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:46.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:46.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:46.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:46.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:47.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:47.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:47.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:47.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:48.102 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:48.103+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:48.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:48.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:49.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:49.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:49.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:49.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:50.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:50.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:50.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:50.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:51.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:51.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:51.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:51.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:52.047 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:52.047+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:52.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:52.487+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:53.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:53.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:53.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:53.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:54.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:54.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:54.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:54.562+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:55.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:55.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:55.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:55.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:56.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:56.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:56.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:56.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:57.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:57.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:57.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:57.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:58.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:58.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:58.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:58.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:59.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:16:59.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:16:59.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:16:59.545+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:00.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:00.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:00.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:00.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:01.091 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:01.091+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:01.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:01.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:02.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:02.080+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:02.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:02.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:03.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:03.109+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:03.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:03.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:04.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:04.141+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:04.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:04.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:05.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:05.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:05.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:05.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:06.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:06.101+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:06.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:06.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:07.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:07.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:07.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:07.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:08.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:08.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:08.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:08.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:09.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:09.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:09.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:09.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:10.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:10.160+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:10.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:10.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:11.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:11.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:11.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:11.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:12.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:12.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:12.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:12.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:13.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:13.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:13.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:13.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:14.121 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:14.122+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:14.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:14.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:15.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:15.099+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:15.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:15.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:16.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:16.090+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:16.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:16.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:17.102 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:17.102+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:17.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:17.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:18.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:18.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:18.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:18.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:19.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:19.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:19.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:19.425+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:20.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:20.129+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:20.423 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:20.423+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:21.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:21.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:21.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:21.414+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:22.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:22.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:22.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:22.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:23.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:23.102+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:23.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:23.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:24.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:24.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:24.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:24.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:25.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:25.030+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:25.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:25.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:26.015 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:26.015+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:26.460 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:26.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:27.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:27.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:27.415 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:27.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:27.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:27.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:28.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:28.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:28.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:28.992+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:29.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:29.367+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:30.011 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:30.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:30.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:30.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:31.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:31.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:31.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:31.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:31.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:31.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:32.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:32.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:32.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:32.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:33.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:33.426+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:33.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:33.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:34.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:34.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:34.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:34.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:35.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:35.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:35.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:35.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:36.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:36.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:36.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:36.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:37.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:37.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:37.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:37.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:38.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:38.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:38.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:38.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:39.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:39.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:39.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:39.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:40.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:40.324+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:40.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:40.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:41.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:41.298+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:41.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:41.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:42.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:42.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:42.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:42.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:43.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:43.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:43.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:43.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:44.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:44.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:44.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:44.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:45.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:45.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:45.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:45.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:46.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:46.335+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:46.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:46.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:47.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:47.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:47.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:47.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:48.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:48.394+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:48.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:48.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:49.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:49.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:49.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:49.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:50.399 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:50.398+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:50.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:50.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:51.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:51.444+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:51.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:51.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:52.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:52.422+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:52.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:52.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:53.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:53.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:53.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:53.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:54.394 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:54.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:54.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:54.736+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:55.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:55.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:55.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:55.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:56.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:56.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:56.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:56.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:57.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:57.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:57.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:57.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:58.408 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:58.407+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:58.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:58.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:59.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:17:59.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:17:59.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:17:59.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:00.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:00.427+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:00.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:00.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:01.399 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:01.399+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:01.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:01.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:02.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:02.384+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:02.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:02.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:03.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:03.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:03.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:03.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:04.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:04.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:04.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:04.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:05.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:05.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:05.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:05.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:06.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:06.375+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:06.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:06.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:07.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:07.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:07.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:07.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:08.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:08.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:08.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:08.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:09.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:09.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:09.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:09.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:10.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:10.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:10.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:10.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:11.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:11.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:11.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:11.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:12.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:12.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:12.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:12.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:13.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:13.270+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:13.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:13.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:14.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:14.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:14.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:14.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:15.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:15.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:15.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:15.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:16.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:16.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:16.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:16.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:17.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:17.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:17.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:17.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:18.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:18.272+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:18.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:18.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:19.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:19.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:20.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:19.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:20.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:20.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:20.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:20.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:21.368 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:21.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:21.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:21.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:22.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:22.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:22.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:22.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:23.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:23.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:23.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:23.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:24.378 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:24.377+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:24.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:24.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:25.403 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:25.402+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:25.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:25.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:26.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:26.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:26.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:26.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:27.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:27.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:27.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:27.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:28.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:28.416+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:28.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:28.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:29.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:29.416+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:29.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:29.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:30.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:30.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:30.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:30.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:31.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:31.444+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:31.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:31.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:32.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:32.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:32.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:32.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:33.451 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:33.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:33.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:33.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:34.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:34.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:34.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:34.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:35.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:35.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:35.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:35.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:36.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:36.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:36.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:36.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:37.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:37.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:37.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:37.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:38.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:38.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:38.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:38.521+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:39.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:39.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:39.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:39.556+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:40.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:40.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:40.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:40.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:41.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:41.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:41.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:41.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:42.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:42.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:42.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:42.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:43.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:43.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:43.562 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:43.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:44.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:44.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:44.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:44.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:45.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:45.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:45.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:45.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:46.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:46.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:46.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:46.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:47.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:47.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:47.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:47.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:48.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:48.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:48.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:48.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:49.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:49.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:49.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:49.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:50.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:50.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:50.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:50.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:51.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:51.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:51.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:51.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:52.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:52.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:52.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:52.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:53.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:53.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:53.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:53.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:54.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:54.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:54.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:54.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:55.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:55.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:55.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:55.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:56.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:56.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:56.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:56.717+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:57.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:57.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:57.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:57.768+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:58.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:58.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:58.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:58.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:59.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:18:59.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:18:59.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:18:59.768+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:00.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:00.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:00.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:00.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:01.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:01.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:01.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:01.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:02.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:02.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:02.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:02.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:03.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:03.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:03.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:03.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:04.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:04.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:04.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:04.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:05.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:05.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:05.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:05.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:06.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:06.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:06.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:06.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:07.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:07.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:07.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:07.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:08.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:08.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:08.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:08.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:09.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:09.390+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:09.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:09.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:10.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:10.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:10.579 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:10.579+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:11.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:11.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:11.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:11.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:12.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:12.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:12.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:12.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:13.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:13.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:13.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:13.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:14.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:14.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:14.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:14.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:15.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:15.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:15.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:15.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:16.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:16.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:16.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:16.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:17.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:17.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:17.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:17.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:18.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:18.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:18.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:18.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:19.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:19.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:19.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:19.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:20.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:20.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:20.508 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:20.508+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:21.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:21.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:21.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:21.514+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:22.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:22.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:22.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:22.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:23.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:23.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:23.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:23.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:24.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:24.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:24.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:24.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:25.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:25.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:25.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:25.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:26.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:26.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:26.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:26.481+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:27.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:27.421+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:27.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:27.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:28.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:28.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:28.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:28.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:29.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:29.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:29.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:29.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:30.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:30.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:30.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:30.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:31.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:31.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:31.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:31.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:32.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:32.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:32.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:32.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:33.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:33.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:33.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:33.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:34.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:34.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:34.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:34.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:35.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:35.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:35.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:35.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:36.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:36.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:36.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:36.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:37.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:37.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:37.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:37.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:38.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:38.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:38.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:38.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:39.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:39.507+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:39.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:39.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:40.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:40.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:40.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:40.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:41.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:41.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:41.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:41.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:42.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:42.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:42.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:42.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:43.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:43.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:43.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:43.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:44.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:44.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:44.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:44.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:45.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:45.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:45.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:45.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:46.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:46.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:46.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:46.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:47.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:47.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:47.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:47.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:48.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:48.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:48.623 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:48.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:49.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:49.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:49.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:49.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:50.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:50.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:50.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:50.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:51.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:51.551+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:51.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:51.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:52.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:52.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:52.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:52.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:53.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:53.542+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:53.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:53.554+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:54.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:54.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:54.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:54.548+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:55.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:55.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:55.501 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:55.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:56.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:56.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:56.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:56.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:57.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:57.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:57.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:57.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:58.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:58.460+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:58.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:58.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:59.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:19:59.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:19:59.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:19:59.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:00.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:00.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:00.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:00.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:01.427 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:01.427+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:01.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:01.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:02.378 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:02.378+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:02.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:02.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:03.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:03.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:03.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:03.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:04.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:04.384+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:04.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:04.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:05.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:05.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:05.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:05.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:06.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:06.335+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:06.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:06.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:07.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:07.288+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:07.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:07.359+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:08.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:08.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:08.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:08.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:09.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:09.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:09.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:09.357+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:10.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:10.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:10.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:10.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:11.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:11.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:11.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:11.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:12.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:12.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:12.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:12.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:13.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:13.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:13.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:13.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:14.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:14.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:14.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:14.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:15.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:15.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:15.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:15.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:16.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:16.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:16.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:16.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:17.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:17.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:17.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:17.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:18.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:18.120+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:18.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:18.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:19.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:19.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:19.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:19.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:20.146 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:20.145+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:20.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:20.166+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:21.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:21.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:21.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:21.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:22.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:22.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:22.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:22.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:23.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:23.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:23.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:23.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:24.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:24.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:24.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:24.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:25.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:25.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:25.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:25.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:26.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:26.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:26.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:26.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:27.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:27.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:27.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:27.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:28.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:28.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:28.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:28.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:29.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:29.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:29.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:29.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:30.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:30.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:30.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:30.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:30.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:30.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:31.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:31.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:31.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:31.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:32.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:32.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:32.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:32.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:33.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:33.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:33.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:33.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:34.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:34.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:34.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:34.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:35.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:35.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:35.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:35.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:36.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:36.257+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:36.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:36.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:37.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:37.274+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:38.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:38.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:38.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:38.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:39.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:39.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:39.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:39.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:40.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:40.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:40.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:40.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:41.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:41.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:41.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:41.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:42.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:42.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:42.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:42.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:43.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:43.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:43.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:43.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:44.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:44.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:44.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:44.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:45.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:45.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:45.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:45.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:46.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:46.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:46.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:46.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:47.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:47.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:47.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:47.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:48.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:48.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:48.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:48.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:49.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:49.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:49.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:49.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:50.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:50.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:50.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:50.158+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:51.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:51.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:51.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:51.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:52.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:52.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:52.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:52.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:53.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:53.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:53.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:53.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:54.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:54.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:54.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:54.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:55.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:55.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:55.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:55.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:56.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:56.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:56.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:56.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:57.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:57.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:57.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:57.258+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:58.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:58.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:58.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:58.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:59.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:20:59.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:20:59.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:20:59.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:00.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:00.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:00.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:00.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:01.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:01.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:01.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:01.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:02.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:02.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:02.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:02.170+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:03.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:03.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:03.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:03.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:04.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:04.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:04.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:04.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:05.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:05.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:05.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:05.154+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:06.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:06.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:06.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:06.158+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:07.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:07.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:07.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:07.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:08.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:08.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:08.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:08.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:09.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:09.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:09.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:09.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:10.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:10.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:10.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:10.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:11.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:11.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:11.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:11.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:12.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:12.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:12.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:12.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:13.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:13.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:13.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:13.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:13.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:13.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:14.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:14.307+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:14.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:14.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:15.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:15.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:16.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:16.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:16.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:16.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:17.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:17.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:17.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:17.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:17.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:17.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:18.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:18.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:19.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:19.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:19.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:19.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:20.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:20.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:20.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:20.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:21.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:21.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:21.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:21.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:22.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:22.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:22.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:22.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:23.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:23.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:23.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:23.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:24.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:24.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:24.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:24.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:25.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:25.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:25.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:25.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:26.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:26.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:26.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:26.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:27.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:27.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:27.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:27.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:28.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:28.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:28.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:28.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:29.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:29.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:29.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:29.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:30.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:30.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:30.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:30.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:31.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:31.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:31.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:31.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:32.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:32.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:32.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:32.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:33.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:33.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:33.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:33.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:34.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:34.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:34.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:34.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:35.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:35.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:35.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:35.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:36.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:36.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:36.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:36.394+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:37.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:37.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:37.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:37.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:38.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:38.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:38.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:38.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:39.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:39.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:39.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:39.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:40.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:40.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:40.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:40.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:41.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:41.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:41.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:41.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:42.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:42.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:42.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:42.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:43.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:43.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:43.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:43.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:44.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:44.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:44.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:44.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:45.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:45.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:45.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:45.357+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:46.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:46.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:46.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:46.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:47.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:47.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:47.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:47.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:48.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:48.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:48.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:48.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:49.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:49.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:49.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:49.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:50.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:50.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:50.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:50.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:51.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:51.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:51.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:51.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:52.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:52.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:52.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:52.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:53.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:53.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:53.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:53.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:54.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:54.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:54.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:54.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:55.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:55.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:55.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:55.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:56.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:56.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:56.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:56.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:57.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:57.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:57.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:57.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:58.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:58.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:58.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:58.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:59.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:21:59.155+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:21:59.395 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:21:59.395+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:00.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:00.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:00.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:00.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:01.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:01.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:01.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:01.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:02.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:02.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:02.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:02.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:03.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:03.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:03.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:03.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:04.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:04.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:04.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:04.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:05.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:05.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:05.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:05.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:06.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:06.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:06.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:06.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:07.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:07.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:07.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:07.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:08.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:08.184+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:08.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:08.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:09.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:09.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:09.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:09.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:10.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:10.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:10.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:10.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:11.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:11.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:11.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:11.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:12.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:12.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:12.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:12.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:13.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:13.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:13.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:13.263+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:14.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:14.145+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:14.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:14.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:15.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:15.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:15.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:15.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:16.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:16.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:16.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:16.165+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:17.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:17.154+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:17.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:17.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:18.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:18.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:18.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:18.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:19.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:19.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:19.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:19.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:20.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:20.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:20.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:20.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:21.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:21.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:21.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:21.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:22.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:22.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:22.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:22.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:23.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:23.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:23.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:23.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:24.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:24.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:24.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:24.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:25.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:25.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:25.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:25.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:26.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:26.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:26.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:26.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:27.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:27.134+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:27.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:27.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:28.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:28.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:28.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:28.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:29.053 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:29.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:29.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:29.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:30.101 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:30.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:30.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:30.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:31.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:31.136+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:31.356 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:31.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:32.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:32.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:32.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:32.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:33.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:33.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:33.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:33.298+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:34.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:34.144+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:34.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:34.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:35.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:35.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:35.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:35.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:36.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:36.123+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:36.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:36.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:37.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:37.139+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:37.355 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:37.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:38.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:38.103+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:38.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:38.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:39.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:39.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:39.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:39.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:40.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:40.106+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:40.351 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:40.351+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:41.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:41.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:41.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:41.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:42.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:42.080+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:42.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:42.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:43.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:43.114+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:43.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:43.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:44.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:44.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:44.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:44.312+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:45.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:45.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:45.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:45.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:46.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:46.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:46.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:46.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:47.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:47.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:47.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:47.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:48.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:48.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:48.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:48.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:49.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:49.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:49.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:49.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:50.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:50.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:50.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:50.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:51.114 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:51.114+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:51.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:51.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:52.148 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:52.149+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:52.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:52.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:53.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:53.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:53.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:53.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:54.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:54.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:54.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:54.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:55.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:55.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:55.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:55.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:56.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:56.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:56.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:56.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:57.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:57.191+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:57.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:57.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:58.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:58.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:58.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:58.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:59.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:22:59.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:22:59.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:22:59.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:00.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:00.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:00.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:00.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:01.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:01.208+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:01.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:01.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:02.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:02.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:02.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:02.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:03.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:03.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:03.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:03.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:04.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:04.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:04.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:04.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:05.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:05.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:05.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:05.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:06.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:06.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:06.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:06.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:07.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:07.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:07.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:07.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:08.130 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:08.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:08.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:08.412+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:09.142 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:09.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:09.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:09.377+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:10.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:10.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:10.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:10.389+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:11.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:11.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:11.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:11.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:12.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:12.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:12.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:12.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:13.138 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:13.139+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:13.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:13.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:14.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:14.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:14.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:14.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:15.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:15.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:15.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:15.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:16.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:16.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:16.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:16.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:17.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:17.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:17.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:17.441+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:18.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:18.028+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:18.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:18.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:19.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:19.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:19.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:19.409+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:20.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:20.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:20.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:20.451+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:20.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:20.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:21.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:21.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:21.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:21.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:22.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:22.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:22.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:22.957+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:23.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:23.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:23.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:23.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:24.470 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:24.470+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:24.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:24.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:25.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:25.443+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:25.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:25.937+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:26.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:26.402+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:26.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:26.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:27.435 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:27.435+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:27.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:27.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:28.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:28.479+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:28.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:28.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:29.459 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:29.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:29.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:29.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:30.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:30.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:30.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:30.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:31.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:31.381+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:31.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:31.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:32.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:32.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:32.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:32.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:33.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:33.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:33.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:33.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:34.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:34.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:34.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:34.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:35.369 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:35.369+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:35.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:35.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:36.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:36.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:36.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:36.973+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:37.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:37.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:37.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:37.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:38.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:38.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:38.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:38.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:39.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:39.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:39.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:39.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:40.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:40.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:40.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:40.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:41.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:41.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:41.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:41.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:42.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:42.361+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:42.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:42.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:43.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:43.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:43.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:43.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:44.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:44.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:44.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:44.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:45.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:45.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:45.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:45.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:46.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:46.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:46.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:46.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:47.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:47.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:47.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:47.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:48.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:48.312+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:48.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:48.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:49.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:49.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:49.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:49.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:50.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:50.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:50.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:50.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:51.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:51.436+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:51.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:51.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:52.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:52.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:52.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:52.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:53.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:53.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:53.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:53.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:54.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:54.400+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:54.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:54.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:55.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:55.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:55.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:55.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:56.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:56.449+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:56.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:56.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:57.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:57.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:57.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:57.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:58.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:58.458+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:58.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:58.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:59.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:23:59.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:23:59.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:23:59.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:00.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:00.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:00.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:00.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:01.490 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:01.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:01.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:01.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:02.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:02.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:02.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:02.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:03.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:03.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:03.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:03.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:04.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:04.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:04.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:04.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:05.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:05.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:05.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:05.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:06.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:06.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:06.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:06.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:07.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:07.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:07.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:07.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:08.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:08.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:08.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:08.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:09.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:09.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:09.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:09.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:10.302 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:10.302+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:10.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:10.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:11.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:11.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:11.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:11.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:12.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:12.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:12.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:12.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:13.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:13.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:13.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:13.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:14.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:14.335+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:14.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:14.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:15.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:15.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:15.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:15.823+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:16.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:16.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:16.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:16.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:17.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:17.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:17.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:17.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:18.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:18.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:18.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:18.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:19.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:19.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:19.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:19.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:20.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:20.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:20.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:20.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:21.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:21.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:21.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:21.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:22.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:22.196+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:22.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:22.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:23.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:23.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:23.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:23.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:24.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:24.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:24.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:24.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:25.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:25.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:25.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:25.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:26.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:26.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:26.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:26.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:27.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:27.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:27.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:27.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:28.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:28.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:28.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:28.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:29.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:29.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:29.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:29.823+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:30.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:30.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:30.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:30.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:31.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:31.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:31.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:31.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:32.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:32.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:32.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:32.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:33.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:33.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:33.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:33.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:34.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:34.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:34.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:34.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:35.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:35.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:35.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:35.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:36.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:36.157+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:36.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:36.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:37.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:37.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:37.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:37.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:38.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:38.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:38.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:38.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:39.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:39.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:39.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:39.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:40.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:40.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:40.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:40.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:41.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:41.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:41.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:41.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:42.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:42.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:42.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:42.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:43.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:43.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:43.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:43.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:44.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:44.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:44.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:44.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:45.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:45.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:45.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:45.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:46.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:46.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:46.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:46.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:47.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:47.316+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:47.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:47.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:48.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:48.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:48.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:48.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:49.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:49.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:49.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:49.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:50.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:50.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:50.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:50.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:51.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:51.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:51.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:51.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:52.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:52.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:52.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:52.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:53.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:53.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:53.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:53.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:54.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:54.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:54.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:54.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:55.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:55.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:55.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:55.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:56.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:56.357+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:56.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:56.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:57.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:57.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:57.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:57.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:58.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:58.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:58.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:58.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:59.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:24:59.367+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:24:59.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:24:59.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:00.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:00.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:00.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:00.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:01.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:01.350+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:01.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:01.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:02.351 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:02.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:02.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:02.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:03.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:03.367+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:03.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:03.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:04.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:04.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:04.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:04.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:05.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:05.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:05.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:05.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:06.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:06.335+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:06.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:06.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:07.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:07.339+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:07.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:07.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:08.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:08.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:08.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:08.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:09.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:09.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:09.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:09.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:10.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:10.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:10.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:10.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:11.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:11.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:11.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:11.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:12.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:12.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:12.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:12.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:13.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:13.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:13.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:13.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:14.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:14.421+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:14.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:14.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:15.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:15.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:15.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:15.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:16.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:16.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:16.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:16.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:17.423 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:17.424+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:17.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:17.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:18.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:18.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:18.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:18.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:19.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:19.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:19.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:19.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:20.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:20.487+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:20.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:20.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:21.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:21.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:21.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:21.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:22.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:22.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:22.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:22.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:23.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:23.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:23.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:23.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:24.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:24.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:24.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:24.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:25.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:25.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:25.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:25.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:26.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:26.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:26.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:26.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:27.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:27.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:27.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:27.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:28.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:28.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:28.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:28.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:29.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:29.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:29.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:29.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:30.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:30.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:30.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:30.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:31.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:31.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:31.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:31.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:32.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:32.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:32.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:32.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:33.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:33.539+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:33.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:33.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:34.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:34.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:34.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:34.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:35.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:35.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:35.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:35.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:36.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:36.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:36.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:36.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:37.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:37.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:37.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:37.961+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:38.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:38.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:38.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:38.982+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:39.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:39.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:39.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:39.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:40.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:40.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:40.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:40.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:41.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:41.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:42.015 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:42.015+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:42.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:42.623+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:42.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:42.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:43.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:43.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:44.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:44.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:44.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:44.695+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:45.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:45.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:45.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:45.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:46.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:46.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:46.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:46.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:47.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:47.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:47.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:47.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:48.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:48.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:48.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:48.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:49.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:49.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:49.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:49.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:50.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:50.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:50.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:50.718+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:51.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:51.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:51.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:51.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:52.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:52.037+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:52.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:52.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:53.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:53.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:53.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:53.739+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:54.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:54.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:54.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:54.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:55.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:55.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:55.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:55.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:56.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:56.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:56.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:56.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:57.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:57.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:57.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:57.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:58.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:58.109+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:58.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:58.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:59.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:25:59.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:25:59.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:25:59.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:00.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:00.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:00.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:00.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:01.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:01.083+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:01.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:01.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:02.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:02.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:02.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:02.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:03.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:03.152+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:03.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:03.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:04.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:04.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:04.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:04.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:05.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:05.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:05.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:05.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:06.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:06.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:06.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:06.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:07.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:07.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:07.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:07.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:08.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:08.059+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:08.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:08.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:09.047 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:09.047+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:09.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:09.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:10.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:10.085+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:10.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:10.799+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:11.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:11.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:11.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:11.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:12.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:12.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:12.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:12.782+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:13.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:13.141+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:13.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:13.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:14.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:14.189+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:14.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:14.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:15.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:15.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:15.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:15.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:16.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:16.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:16.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:16.860+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:17.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:17.218+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:17.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:17.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:18.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:18.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:18.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:18.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:19.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:19.203+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:19.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:19.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:20.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:20.210+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:20.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:20.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:21.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:21.202+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:21.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:21.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:22.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:22.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:22.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:22.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:23.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:23.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:23.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:23.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:24.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:24.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:24.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:24.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:25.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:25.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:25.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:25.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:26.235 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:26.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:26.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:26.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:27.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:27.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:27.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:27.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:28.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:28.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:28.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:28.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:29.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:29.274+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:29.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:29.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:30.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:30.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:30.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:30.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:31.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:31.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:31.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:31.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:32.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:32.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:32.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:32.933+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:33.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:33.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:33.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:33.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:34.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:34.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:34.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:34.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:35.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:35.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:35.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:35.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:36.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:36.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:36.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:36.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:37.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:37.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:37.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:37.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:38.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:38.361+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:38.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:38.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:39.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:39.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:39.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:39.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:40.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:40.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:41.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:41.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:41.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:41.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:42.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:42.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:42.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:42.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:43.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:43.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:43.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:43.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:44.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:44.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:44.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:44.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:44.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:44.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:45.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:45.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:45.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:45.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:46.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:46.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:46.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:46.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:47.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:47.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:47.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:47.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:48.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:48.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:48.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:48.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:49.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:49.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:49.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:49.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:50.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:50.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:50.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:50.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:51.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:51.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:51.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:51.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:52.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:52.257+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:52.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:52.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:53.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:53.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:53.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:53.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:54.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:54.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:54.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:54.848+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:55.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:55.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:55.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:55.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:56.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:56.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:56.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:56.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:57.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:57.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:57.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:57.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:58.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:58.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:58.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:58.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:59.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:26:59.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:26:59.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:26:59.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:00.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:00.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:00.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:00.786+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:01.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:01.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:01.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:01.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:02.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:02.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:02.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:02.786+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:03.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:03.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:03.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:03.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:04.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:04.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:04.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:04.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:05.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:05.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:05.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:05.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:06.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:06.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:06.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:06.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:07.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:07.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:07.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:07.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:08.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:08.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:08.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:08.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:09.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:09.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:09.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:09.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:10.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:10.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:10.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:10.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:11.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:11.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:11.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:11.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:12.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:12.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:12.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:12.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:13.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:13.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:13.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:13.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:14.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:14.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:14.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:14.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:15.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:15.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:15.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:15.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:16.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:16.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:16.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:16.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:17.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:17.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:17.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:17.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:18.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:18.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:18.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:18.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:19.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:19.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:19.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:19.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:20.491 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:20.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:20.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:20.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:21.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:21.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:21.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:21.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:22.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:22.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:22.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:22.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:23.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:23.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:23.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:23.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:24.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:24.439+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:24.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:24.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:25.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:25.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:25.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:25.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:26.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:26.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:26.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:26.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:27.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:27.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:27.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:27.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:28.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:28.409+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:28.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:28.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:29.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:29.376+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:29.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:29.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:30.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:30.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:30.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:30.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:31.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:31.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:31.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:31.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:32.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:32.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:32.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:32.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:33.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:33.372+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:33.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:33.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:34.364 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:34.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:34.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:34.676+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:35.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:35.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:35.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:35.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:36.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:36.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:36.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:36.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:37.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:37.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:37.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:37.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:38.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:38.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:38.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:38.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:39.412 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:39.411+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:39.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:39.701+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:40.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:40.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:40.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:40.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:41.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:41.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:41.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:41.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:42.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:42.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:42.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:42.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:43.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:43.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:43.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:43.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:44.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:44.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:44.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:44.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:45.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:45.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:45.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:45.449+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:46.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:46.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:46.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:46.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:47.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:47.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:47.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:47.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:48.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:48.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:48.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:48.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:49.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:49.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:49.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:49.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:50.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:50.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:50.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:50.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:51.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:51.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:51.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:51.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:52.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:52.381+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:52.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:52.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:53.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:53.361+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:53.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:53.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:54.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:54.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:54.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:54.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:55.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:55.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:55.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:55.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:56.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:56.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:56.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:56.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:57.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:57.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:57.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:57.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:58.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:58.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:58.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:58.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:59.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:27:59.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:27:59.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:27:59.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:00.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:00.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:00.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:00.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:01.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:01.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:01.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:01.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:02.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:02.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:02.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:02.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:03.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:03.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:03.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:03.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:04.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:04.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:04.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:04.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:05.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:05.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:05.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:05.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:06.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:06.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:06.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:06.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:07.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:07.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:07.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:07.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:08.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:08.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:08.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:08.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:09.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:09.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:09.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:09.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:10.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:10.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:10.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:10.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:11.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:11.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:11.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:11.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:12.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:12.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:12.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:12.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:13.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:13.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:13.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:13.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:14.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:14.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:14.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:14.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:15.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:15.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:16.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:16.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:16.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:16.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:17.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:17.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:17.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:17.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:18.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:18.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:18.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:18.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:19.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:19.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:19.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:19.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:20.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:20.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:20.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:20.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:21.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:21.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:21.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:21.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:22.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:22.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:22.412 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:22.412+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:23.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:23.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:23.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:23.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:24.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:24.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:24.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:24.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:25.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:25.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:25.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:25.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:26.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:26.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:26.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:26.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:27.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:27.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:27.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:27.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:28.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:28.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:28.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:28.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:29.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:29.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:29.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:29.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:30.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:30.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:30.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:30.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:31.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:31.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:31.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:31.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:31.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:31.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:32.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:32.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:32.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:32.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:33.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:33.439+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:33.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:33.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:34.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:34.478+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:34.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:34.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:35.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:35.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:35.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:35.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:36.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:36.478+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:36.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:36.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:37.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:37.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:37.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:37.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:38.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:38.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:38.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:38.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:39.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:39.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:39.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:39.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:40.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:40.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:40.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:40.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:41.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:41.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:41.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:41.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:42.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:42.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:42.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:42.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:43.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:43.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:43.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:43.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:44.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:44.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:44.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:44.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:45.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:45.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:45.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:45.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:46.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:46.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:46.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:46.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:47.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:47.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:47.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:47.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:48.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:48.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:48.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:48.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:49.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:49.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:49.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:49.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:50.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:50.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:50.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:50.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:51.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:51.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:51.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:51.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:52.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:52.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:52.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:52.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:53.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:53.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:53.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:53.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:54.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:54.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:54.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:54.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:55.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:55.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:55.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:55.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:56.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:56.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:56.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:56.784+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:57.448 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:57.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:57.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:57.775+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:58.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:58.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:58.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:58.781+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:59.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:28:59.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:28:59.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:28:59.745+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:00.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:00.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:00.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:00.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:01.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:01.412+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:01.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:01.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:02.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:02.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:02.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:02.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:03.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:03.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:03.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:03.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:04.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:04.404+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:04.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:04.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:05.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:05.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:05.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:05.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:06.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:06.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:06.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:06.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:07.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:07.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:07.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:07.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:08.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:08.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:08.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:08.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:09.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:09.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:09.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:09.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:10.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:10.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:10.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:10.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:11.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:11.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:11.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:11.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:12.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:12.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:12.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:12.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:13.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:13.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:13.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:13.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:14.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:14.411+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:14.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:14.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:15.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:15.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:15.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:15.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:16.335 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:16.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:16.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:16.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:17.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:17.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:17.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:17.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:18.344 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:18.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:18.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:18.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:19.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:19.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:19.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:19.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:20.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:20.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:20.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:20.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:21.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:21.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:21.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:21.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:22.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:22.277+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:22.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:22.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:23.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:23.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:23.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:23.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:24.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:24.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:24.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:24.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:25.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:25.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:25.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:25.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:26.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:26.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:26.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:26.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:27.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:27.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:27.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:27.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:28.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:28.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:28.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:28.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:29.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:29.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:29.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:29.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:30.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:30.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:30.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:30.781+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:31.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:31.380+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:31.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:31.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:32.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:32.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:32.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:32.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:33.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:33.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:33.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:33.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:34.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:34.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:34.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:34.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:35.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:35.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:35.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:35.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:36.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:36.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:36.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:36.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:37.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:37.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:37.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:37.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:38.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:38.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:38.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:38.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:39.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:39.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:39.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:39.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:40.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:40.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:40.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:40.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:41.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:41.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:41.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:41.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:42.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:42.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:42.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:42.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:43.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:43.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:43.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:43.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:44.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:44.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:44.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:44.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:45.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:45.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:46.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:46.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:46.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:46.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:46.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:46.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:47.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:47.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:47.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:47.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:48.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:48.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:49.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:49.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:49.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:49.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:50.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:50.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:50.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:50.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:50.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:50.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:51.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:51.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:51.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:51.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:52.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:52.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:52.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:52.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:53.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:53.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:53.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:53.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:54.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:54.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:54.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:54.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:55.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:55.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:55.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:55.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:56.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:56.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:56.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:56.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:57.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:57.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:57.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:57.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:58.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:58.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:58.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:58.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:59.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:29:59.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:29:59.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:29:59.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:00.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:00.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:00.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:00.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:01.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:01.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:01.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:01.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:02.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:02.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:02.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:02.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:03.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:03.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:03.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:03.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:04.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:04.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:04.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:04.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:05.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:05.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:05.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:05.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:06.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:06.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:06.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:06.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:07.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:07.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:07.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:07.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:08.228 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T20:30:08.227+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffb6c034140 msgr2=0x7ffb6c047380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T20:30:08.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:08.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:08.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:08.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:09.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:09.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:09.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:09.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:10.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:10.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:10.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:10.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:11.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:11.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:12.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:12.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:12.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:12.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:13.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:13.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:13.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:13.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:14.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:14.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:14.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:14.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:15.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:15.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:15.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:15.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:16.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:16.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:16.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:16.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:17.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:17.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:17.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:17.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:18.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:18.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:18.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:18.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:19.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:19.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:19.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:19.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:20.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:20.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:20.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:20.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:21.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:21.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:21.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:21.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:22.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:22.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:22.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:22.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:23.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:23.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:23.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:23.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:24.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:24.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:24.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:24.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:25.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:25.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:25.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:25.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:26.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:26.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:26.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:26.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:27.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:27.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:27.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:27.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:28.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:28.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:28.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:28.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:29.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:29.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:29.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:29.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:30.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:30.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:30.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:30.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:31.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:31.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:31.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:31.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:32.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:32.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:32.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:32.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:33.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:33.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:33.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:33.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:34.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:34.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:34.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:34.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:35.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:35.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:35.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:35.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:36.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:36.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:36.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:36.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:37.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:37.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:37.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:37.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:37.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:37.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:38.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:38.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:39.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:39.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:39.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:39.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:40.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:40.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:40.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:40.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:41.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:41.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:41.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:41.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:42.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:42.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:42.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:42.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:42.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:42.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:43.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:43.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:43.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:43.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:44.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:44.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:44.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:44.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:45.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:45.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:46.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:46.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:46.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:46.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:46.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:46.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:47.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:47.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:47.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:47.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:48.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:48.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:48.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:48.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:49.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:49.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:49.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:49.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:50.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:50.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:50.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:50.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:51.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:51.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:51.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:51.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:52.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:52.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:52.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:52.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:53.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:53.504+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:53.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:53.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:54.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:54.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:54.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:54.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:55.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:55.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:55.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:55.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:56.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:56.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:56.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:56.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:57.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:57.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:57.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:57.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:58.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:58.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:58.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:58.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:59.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:30:59.551+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:30:59.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:30:59.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:00.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:00.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:01.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:01.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:01.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:01.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:02.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:02.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:02.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:02.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:03.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:03.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:03.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:03.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:03.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:03.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:04.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:04.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:04.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:04.933+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:05.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:05.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:05.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:05.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:06.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:06.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:06.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:06.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:07.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:07.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:07.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:07.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:08.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:08.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:08.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:08.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:09.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:09.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:09.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:09.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:10.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:10.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:10.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:10.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:11.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:11.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:11.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:11.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:12.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:12.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:13.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:12.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:13.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:13.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:13.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:13.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:14.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:14.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:14.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:14.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:15.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:15.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:15.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:15.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:16.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:16.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:16.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:16.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:17.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:17.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:17.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:17.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:18.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:18.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:18.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:18.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:19.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:19.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:19.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:19.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:20.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:20.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:20.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:20.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:21.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:21.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:21.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:21.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:22.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:22.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:22.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:22.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:23.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:23.504+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:23.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:23.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:24.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:24.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:24.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:24.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:25.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:25.435+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:25.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:25.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:26.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:26.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:26.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:26.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:27.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:27.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:27.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:27.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:28.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:28.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:28.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:28.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:29.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:29.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:29.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:29.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:30.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:30.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:30.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:30.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:31.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:31.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:31.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:31.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:32.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:32.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:32.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:32.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:33.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:33.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:34.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:34.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:34.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:34.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:35.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:35.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:35.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:35.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:36.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:36.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:36.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:36.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:37.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:37.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:37.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:37.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:38.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:38.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:38.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:38.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:38.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:38.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:39.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:39.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:39.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:39.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:40.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:40.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:40.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:40.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:41.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:41.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:41.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:41.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:42.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:42.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:42.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:42.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:43.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:43.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:43.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:43.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:44.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:44.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:44.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:44.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:45.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:45.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:45.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:45.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:46.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:46.539+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:46.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:46.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:47.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:47.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:47.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:47.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:48.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:48.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:49.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:49.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:49.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:49.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:50.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:50.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:50.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:50.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:50.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:50.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:51.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:51.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:52.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:52.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:52.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:52.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:53.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:53.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:53.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:53.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:53.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:53.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:54.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:54.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:54.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:54.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:55.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:55.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:56.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:56.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:56.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:56.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:57.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:57.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:57.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:57.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:58.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:58.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:58.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:58.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:59.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:31:59.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:31:59.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:31:59.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:00.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:00.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:00.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:00.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:01.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:01.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:01.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:01.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:02.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:02.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:02.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:02.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:03.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:03.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:03.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:03.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:04.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:04.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:04.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:04.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:05.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:05.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:05.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:05.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:05.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:05.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:06.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:06.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:06.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:06.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:07.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:07.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:08.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:08.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:08.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:08.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:09.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:09.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:09.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:09.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:10.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:10.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:10.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:10.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:11.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:11.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:11.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:11.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:12.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:12.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:12.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:12.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:13.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:13.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:13.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:13.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:14.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:14.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:14.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:14.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:15.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:15.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:15.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:15.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:16.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:16.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:16.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:16.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:16.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:16.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:17.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:17.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:17.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:17.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:18.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:18.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:18.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:18.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:19.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:19.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:19.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:19.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:20.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:20.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:20.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:20.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:21.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:21.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:21.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:21.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:22.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:22.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:22.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:22.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:23.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:23.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:23.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:23.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:24.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:24.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:24.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:24.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:25.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:25.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:25.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:25.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:26.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:26.621+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:26.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:26.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:27.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:27.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:27.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:27.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:28.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:28.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:28.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:28.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:29.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:29.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:29.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:29.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:30.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:30.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:30.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:30.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:31.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:31.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:31.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:31.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:32.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:32.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:32.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:32.781+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:33.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:33.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:33.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:33.758+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:34.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:34.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:34.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:34.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:35.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:35.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:35.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:35.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:36.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:36.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:36.837 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:36.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:37.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:37.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:37.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:37.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:38.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:38.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:38.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:38.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:39.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:39.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:39.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:39.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:40.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:40.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:40.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:40.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:41.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:41.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:41.863 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:41.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:42.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:42.545+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:42.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:42.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:43.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:43.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:43.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:43.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:44.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:44.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:44.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:44.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:45.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:45.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:45.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:45.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:46.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:46.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:46.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:46.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:47.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:47.618+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:47.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:47.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:48.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:48.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:48.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:48.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:49.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:49.580+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:49.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:49.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:50.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:50.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:50.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:50.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:51.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:51.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:51.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:51.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:52.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:52.610+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:52.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:52.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:53.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:53.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:53.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:53.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:54.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:54.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:54.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:54.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:55.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:55.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:55.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:55.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:56.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:56.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:56.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:56.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:57.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:57.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:57.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:57.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:58.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:58.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:58.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:58.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:59.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:32:59.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:32:59.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:32:59.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:00.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:00.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:00.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:00.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:01.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:01.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:01.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:01.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:02.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:02.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:02.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:02.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:03.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:03.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:03.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:03.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:04.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:04.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:05.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:05.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:05.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:05.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:06.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:06.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:06.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:06.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:07.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:07.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:07.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:07.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:08.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:08.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:08.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:08.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:09.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:09.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:09.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:09.435+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:10.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:10.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:10.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:10.439+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:11.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:11.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:11.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:11.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:12.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:12.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:12.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:12.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:13.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:13.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:13.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:13.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:14.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:14.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:14.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:14.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:15.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:15.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:15.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:15.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:16.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:16.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:16.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:16.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:17.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:17.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:17.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:17.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:18.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:18.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:18.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:18.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:19.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:19.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:19.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:19.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:20.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:20.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:20.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:20.354+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:21.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:21.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:21.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:21.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:22.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:22.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:22.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:22.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:23.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:23.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:23.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:23.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:24.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:24.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:24.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:24.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:25.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:25.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:25.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:25.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:26.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:26.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:26.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:26.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:27.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:27.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:27.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:27.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:27.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:27.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:28.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:28.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:29.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:29.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:29.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:29.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:30.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:30.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:30.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:30.394+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:30.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:30.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:31.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:31.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:31.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:31.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:32.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:32.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:32.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:32.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:33.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:33.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:33.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:33.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:34.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:34.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:34.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:34.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:35.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:35.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:35.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:35.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:36.394 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:36.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:36.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:36.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:37.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:37.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:37.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:37.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:38.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:38.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:38.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:38.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:39.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:39.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:39.764 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:39.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:40.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:40.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:40.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:40.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:41.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:41.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:41.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:41.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:42.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:42.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:42.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:42.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:43.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:43.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:43.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:43.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:44.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:44.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:44.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:44.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:45.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:45.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:45.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:45.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:46.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:46.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:46.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:46.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:47.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:47.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:47.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:47.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:48.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:48.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:48.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:48.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:49.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:49.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:49.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:49.933+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:50.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:50.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:50.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:50.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:51.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:51.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:51.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:51.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:52.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:52.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:52.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:52.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:53.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:53.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:54.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:54.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:54.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:54.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:54.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:54.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:55.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:55.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:55.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:55.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:56.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:56.504+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:56.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:56.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:57.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:57.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:57.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:57.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:58.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:58.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:58.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:58.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:59.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:33:59.485+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:33:59.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:33:59.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:00.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:00.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:01.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:01.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:01.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:01.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:02.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:02.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:02.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:02.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:03.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:03.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:03.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:03.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:04.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:04.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:04.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:04.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:05.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:05.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:05.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:05.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:06.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:06.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:06.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:06.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:07.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:07.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:07.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:07.566+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:08.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:08.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:08.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:08.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:09.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:09.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:09.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:09.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:10.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:10.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:10.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:10.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:11.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:11.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:11.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:11.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:12.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:12.166+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:12.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:12.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:13.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:13.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:13.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:13.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:14.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:14.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:14.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:14.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:15.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:15.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:15.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:15.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:16.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:16.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:16.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:16.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:17.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:17.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:17.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:17.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:18.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:18.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:18.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:18.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:19.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:19.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:19.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:19.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:20.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:20.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:20.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:20.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:21.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:21.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:21.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:21.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:22.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:22.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:22.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:22.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:23.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:23.266+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:23.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:23.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:24.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:24.270+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:24.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:24.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:25.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:25.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:25.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:25.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:26.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:26.260+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:26.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:26.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:27.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:27.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:27.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:27.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:28.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:28.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:28.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:28.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:29.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:29.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:29.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:29.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:30.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:30.377+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:30.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:30.485+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:31.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:31.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:31.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:31.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:32.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:32.309+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:32.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:32.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:33.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:33.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:33.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:33.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:34.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:34.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:34.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:34.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:35.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:35.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:35.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:35.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:36.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:36.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:36.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:36.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:37.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:37.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:37.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:37.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:38.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:38.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:38.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:38.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:39.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:39.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:39.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:39.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:40.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:40.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:40.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:40.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:41.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:41.381+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:41.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:41.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:42.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:42.421+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:42.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:42.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:43.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:43.456+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:43.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:43.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:44.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:44.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:44.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:44.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:45.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:45.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:45.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:45.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:46.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:46.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:46.543 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:46.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:47.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:47.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:47.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:47.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:48.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:48.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:48.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:48.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:49.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:49.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:49.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:49.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:50.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:50.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:50.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:50.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:51.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:51.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:51.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:51.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:52.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:52.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:52.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:52.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:53.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:53.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:53.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:53.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:54.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:54.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:54.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:54.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:55.547 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:55.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:55.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:55.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:56.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:56.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:56.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:56.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:57.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:57.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:57.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:57.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:58.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:58.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:58.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:58.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:59.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:34:59.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:34:59.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:34:59.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:00.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:00.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:00.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:00.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:01.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:01.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:01.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:01.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:02.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:02.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:02.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:02.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:03.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:03.485+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:03.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:03.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:04.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:04.534+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:04.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:04.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:05.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:05.567+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:05.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:05.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:06.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:06.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:06.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:06.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:07.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:07.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:07.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:07.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:08.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:08.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:08.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:08.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:09.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:09.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:09.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:09.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:10.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:10.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:10.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:10.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:11.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:11.598+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:11.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:11.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:12.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:12.605+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:12.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:12.965+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:13.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:13.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:13.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:13.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:14.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:14.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:14.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:14.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:15.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:15.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:15.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:15.996+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:16.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:16.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:16.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:16.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:17.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:17.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:17.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:17.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:18.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:18.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:18.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:18.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:19.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:19.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:19.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:19.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:20.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:20.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:20.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:20.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:21.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:21.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:21.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:21.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:22.495 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:22.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:22.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:22.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:23.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:23.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:23.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:23.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:24.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:24.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:24.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:24.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:25.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:25.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:25.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:25.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:26.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:26.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:26.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:26.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:27.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:27.441+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:27.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:27.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:28.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:28.470+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:28.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:28.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:29.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:29.484+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:29.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:29.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:30.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:30.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:30.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:30.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:31.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:31.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:31.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:31.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:32.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:32.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:32.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:32.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:33.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:33.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:33.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:33.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:34.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:34.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:34.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:34.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:35.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:35.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:35.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:35.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:36.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:36.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:36.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:36.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:37.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:37.606+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:37.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:37.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:38.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:38.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:38.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:38.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:39.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:39.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:39.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:39.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:40.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:40.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:40.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:40.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:41.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:41.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:41.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:41.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:42.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:42.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:42.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:42.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:43.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:43.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:43.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:43.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:44.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:44.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:44.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:44.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:45.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:45.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:45.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:45.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:46.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:46.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:46.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:46.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:47.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:47.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:47.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:47.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:48.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:48.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:48.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:48.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:49.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:49.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:49.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:49.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:50.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:50.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:50.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:50.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:51.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:51.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:51.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:51.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:52.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:52.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:52.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:52.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:53.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:53.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:53.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:53.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:54.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:54.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:54.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:54.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:55.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:55.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:55.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:55.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:56.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:56.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:56.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:56.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:57.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:57.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:57.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:57.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:58.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:58.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:58.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:58.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:59.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:35:59.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:35:59.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:35:59.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:00.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:00.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:00.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:00.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:01.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:01.645+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:01.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:01.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:02.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:02.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:02.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:02.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:03.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:03.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:03.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:03.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:04.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:04.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:04.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:04.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:05.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:05.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:05.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:05.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:06.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:06.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:06.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:06.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:07.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:07.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:07.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:07.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:08.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:08.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:08.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:08.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:09.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:09.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:09.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:09.761+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:10.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:10.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:10.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:10.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:11.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:11.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:11.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:11.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:12.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:12.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:12.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:12.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:13.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:13.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:13.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:13.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:14.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:14.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:14.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:14.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:15.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:15.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:15.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:15.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:16.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:16.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:16.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:16.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:17.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:17.622+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:17.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:17.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:18.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:18.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:18.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:18.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:19.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:19.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:19.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:19.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:20.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:20.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:20.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:20.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:21.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:21.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:21.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:21.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:22.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:22.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:22.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:22.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:23.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:23.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:23.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:23.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:24.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:24.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:24.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:24.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:25.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:25.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:25.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:25.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:26.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:26.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:26.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:26.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:27.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:27.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:27.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:27.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:28.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:28.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:28.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:28.743+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:29.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:29.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:29.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:29.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:30.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:30.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:30.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:30.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:31.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:31.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:31.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:31.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:32.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:32.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:32.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:32.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:33.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:33.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:33.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:33.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:34.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:34.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:34.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:34.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:35.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:35.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:35.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:35.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:36.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:36.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:36.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:36.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:37.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:37.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:37.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:37.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:38.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:38.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:38.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:38.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:39.603 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:39.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:39.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:39.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:40.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:40.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:40.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:40.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:41.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:41.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:41.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:41.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:42.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:42.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:42.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:42.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:43.516 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:43.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:43.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:43.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:44.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:44.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:44.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:44.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:45.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:45.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:45.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:45.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:46.495 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:46.495+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:46.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:46.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:47.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:47.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:47.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:47.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:48.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:48.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:48.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:48.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:49.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:49.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:49.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:49.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:50.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:50.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:50.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:50.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:51.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:51.441+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:51.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:51.867+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:52.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:52.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:52.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:52.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:53.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:53.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:53.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:53.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:54.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:54.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:54.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:54.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:55.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:55.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:55.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:55.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:56.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:56.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:56.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:56.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:57.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:57.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:57.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:57.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:58.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:58.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:58.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:58.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:59.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:36:59.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:36:59.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:36:59.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:00.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:00.577+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:00.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:00.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:01.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:01.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:01.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:01.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:02.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:02.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:02.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:02.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:03.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:03.561+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:03.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:03.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:04.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:04.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:04.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:04.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:05.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:05.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:05.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:05.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:06.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:06.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:06.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:06.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:07.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:07.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:07.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:07.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:08.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:08.530+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:08.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:08.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:09.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:09.500+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:09.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:09.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:10.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:10.549+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:10.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:10.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:11.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:11.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:11.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:11.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:12.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:12.606+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:12.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:12.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:13.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:13.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:13.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:13.751+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:14.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:14.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:14.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:14.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:15.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:15.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:15.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:15.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:16.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:16.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:16.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:16.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:17.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:17.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:17.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:17.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:18.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:18.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:18.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:18.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:19.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:19.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:19.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:19.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:20.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:20.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:20.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:20.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:21.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:21.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:21.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:21.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:22.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:22.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:22.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:22.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:23.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:23.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:23.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:23.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:24.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:24.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:24.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:24.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:25.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:25.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:25.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:25.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:26.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:26.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:26.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:26.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:27.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:27.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:27.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:27.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:28.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:28.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:28.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:28.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:29.505 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:29.504+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:29.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:29.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:30.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:30.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:30.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:30.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:31.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:31.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:31.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:31.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:32.491 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:32.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:32.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:32.987+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:33.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:33.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:33.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:33.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:34.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:34.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:35.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:35.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:35.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:35.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:36.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:36.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:36.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:36.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:37.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:37.016+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:37.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:37.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:38.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:38.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:38.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:38.595+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:39.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:39.055+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:39.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:39.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:40.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:40.103+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:40.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:40.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:41.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:41.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:41.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:41.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:42.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:42.183+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:42.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:42.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:43.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:43.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:43.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:43.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:44.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:44.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:44.571 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:44.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:45.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:45.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:45.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:45.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:46.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:46.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:46.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:46.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:47.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:47.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:47.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:47.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:48.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:48.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:48.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:48.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:49.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:49.301+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:49.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:49.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:50.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:50.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:50.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:50.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:51.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:51.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:51.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:51.678+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:52.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:52.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:52.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:52.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:53.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:53.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:53.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:53.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:54.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:54.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:54.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:54.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:55.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:55.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:55.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:55.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:56.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:56.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:56.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:56.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:57.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:57.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:57.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:57.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:58.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:58.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:58.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:58.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:59.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:37:59.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:37:59.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:37:59.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:00.152 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:00.152+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:00.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:00.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:01.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:01.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:01.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:01.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:02.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:02.142+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:02.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:02.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:03.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:03.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:03.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:03.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:04.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:04.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:04.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:04.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:05.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:05.073+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:05.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:05.695+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:06.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:06.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:06.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:06.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:07.138 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:07.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:07.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:07.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:08.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:08.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:08.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:08.673+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:09.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:09.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:09.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:09.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:10.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:10.082+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:10.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:10.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:11.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:11.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:11.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:11.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:11.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:11.995+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:12.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:12.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:13.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:13.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:13.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:13.701+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:13.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:13.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:14.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:14.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:14.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:14.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:15.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:15.749+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:15.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:15.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:16.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:16.790+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:16.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:16.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:17.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:17.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:17.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:17.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:18.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:18.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:19.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:19.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:19.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:19.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:20.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:20.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:20.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:20.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:21.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:21.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:21.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:21.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:21.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:21.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:22.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:22.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:23.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:23.016+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:24.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:24.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:24.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:24.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:25.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:25.019+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:25.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:25.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:26.003 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:26.003+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:26.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:26.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:26.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:26.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:27.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:27.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:27.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:27.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:28.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:28.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:29.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:29.028+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:29.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:29.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:29.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:29.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:30.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:30.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:31.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:31.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:31.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:31.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:32.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:32.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:32.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:32.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:32.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:32.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:33.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:33.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:33.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:33.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:34.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:34.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:34.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:34.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:35.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:35.008+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:35.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:35.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:36.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:36.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:36.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:36.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:36.956 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:36.957+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:37.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:37.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:37.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:37.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:38.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:38.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:38.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:38.979+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:39.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:39.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:39.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:39.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:40.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:40.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:41.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:41.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:41.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:41.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:42.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:42.057+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:42.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:42.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:43.041 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:43.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:43.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:43.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:44.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:44.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:44.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:44.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:45.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:45.000+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:45.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:45.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:46.041 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:46.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:46.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:46.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:47.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:47.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:47.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:47.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:48.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:48.034+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:48.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:48.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:49.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:49.000+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:49.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:49.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:50.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:50.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:50.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:50.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:51.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:51.028+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:51.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:51.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:52.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:52.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:52.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:52.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:52.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:52.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:53.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:53.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:54.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:54.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:54.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:54.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:55.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:55.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:55.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:55.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:56.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:56.014+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:56.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:56.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:57.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:57.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:57.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:57.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:57.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:57.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:58.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:58.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:58.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:38:58.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:38:59.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:38:59.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:00.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:00.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:00.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:00.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:00.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:00.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:01.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:01.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:01.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:01.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:02.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:02.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:02.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:02.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:03.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:03.784+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:03.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:03.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:04.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:04.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:04.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:04.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:05.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:05.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:05.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:05.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:06.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:06.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:06.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:06.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:07.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:07.711+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:07.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:07.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:08.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:08.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:08.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:08.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:09.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:09.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:09.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:09.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:10.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:10.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:10.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:10.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:11.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:11.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:11.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:11.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:12.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:12.749+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:12.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:12.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:13.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:13.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:13.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:13.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:14.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:14.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:14.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:14.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:15.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:15.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:15.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:15.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:16.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:16.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:16.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:16.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:17.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:17.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:17.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:17.856+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:18.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:18.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:18.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:18.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:19.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:19.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:19.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:19.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:20.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:20.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:20.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:20.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:21.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:21.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:21.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:21.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:22.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:22.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:22.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:22.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:23.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:23.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:23.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:23.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:24.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:24.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:24.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:24.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:25.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:25.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:25.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:25.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:26.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:26.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:26.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:26.910+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:27.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:27.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:27.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:27.961+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:28.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:28.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:28.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:28.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:29.762 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:29.762+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:29.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:29.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:30.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:30.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:30.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:30.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:31.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:31.798+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:32.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:32.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:32.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:32.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:33.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:33.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:33.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:33.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:34.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:34.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:34.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:34.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:34.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:34.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:35.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:35.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:35.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:35.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:36.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:36.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:37.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:37.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:37.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:37.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:38.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:38.007+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:38.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:38.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:39.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:39.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:39.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:39.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:40.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:40.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:40.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:40.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:41.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:41.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:41.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:41.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:42.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:42.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:42.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:42.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:43.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:43.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:43.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:43.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:44.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:44.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:44.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:44.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:45.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:45.034+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:45.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:45.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:46.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:46.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:46.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:46.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:47.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:47.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:47.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:47.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:48.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:48.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:48.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:48.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:49.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:49.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:49.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:49.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:49.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:49.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:50.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:50.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:50.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:50.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:51.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:51.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:51.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:51.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:52.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:52.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:52.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:52.982+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:53.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:53.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:53.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:53.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:54.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:54.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:54.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:54.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:55.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:55.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:55.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:55.952+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:56.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:56.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:56.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:56.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:57.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:57.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:57.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:57.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:58.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:58.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:58.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:58.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:59.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:39:59.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:39:59.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:39:59.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:00.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:00.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:00.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:00.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:01.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:01.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:01.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:01.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:02.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:02.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:02.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:02.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:03.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:03.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:03.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:03.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:04.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:04.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:04.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:04.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:05.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:05.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:05.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:05.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:06.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:06.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:06.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:06.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:07.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:07.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:08.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:07.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:08.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:08.978+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:09.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:09.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:09.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:09.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:10.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:10.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:10.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:10.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:11.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:11.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:11.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:11.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:12.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:12.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:12.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:12.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:13.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:13.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:13.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:13.996+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:14.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:14.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:14.993 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:14.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:15.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:15.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:16.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:16.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:16.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:16.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:17.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:17.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:17.064 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:17.063+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:18.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:18.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:18.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:18.108+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:19.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:19.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:19.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:19.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:20.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:20.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:20.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:20.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:21.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:21.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:21.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:21.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:22.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:22.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:22.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:22.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:23.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:23.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:23.152 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:23.152+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:24.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:24.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:24.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:24.170+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:24.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:24.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:25.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:25.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:26.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:26.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:26.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:26.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:27.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:27.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:27.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:27.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:28.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:28.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:28.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:28.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:29.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:29.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:29.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:29.108+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:30.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:30.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:30.145 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:30.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:31.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:31.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:31.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:31.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:32.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:32.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:32.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:32.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:33.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:33.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:33.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:33.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:34.053 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:34.054+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:34.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:34.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:35.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:35.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:35.095 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:35.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:36.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:36.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:36.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:36.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:37.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:37.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:37.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:37.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:38.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:38.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:38.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:38.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:39.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:39.087+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:39.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:39.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:40.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:40.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:40.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:40.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:41.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:41.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:41.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:41.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:42.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:42.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:42.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:42.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:43.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:43.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:43.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:43.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:44.095 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:44.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:44.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:44.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:45.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:45.116+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:45.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:45.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:46.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:46.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:46.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:46.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:47.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:47.069+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:47.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:47.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:48.057 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:48.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:48.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:48.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:49.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:49.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:49.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:49.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:50.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:50.113+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:50.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:50.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:51.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:51.122+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:51.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:51.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:52.152 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:52.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:52.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:52.196+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:53.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:53.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:53.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:53.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:54.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:54.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:54.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:54.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:55.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:55.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:55.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:55.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:56.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:56.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:56.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:56.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:57.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:57.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:57.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:57.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:58.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:58.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:58.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:58.283+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:59.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:40:59.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:40:59.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:40:59.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:00.328 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:00.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:00.347 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:00.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:01.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:01.309+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:01.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:01.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:02.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:02.263+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:02.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:02.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:03.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:03.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:03.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:03.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:04.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:04.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:04.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:04.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:05.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:05.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:05.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:05.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:06.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:06.248+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:06.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:06.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:07.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:07.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:07.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:07.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:08.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:08.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:08.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:08.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:09.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:09.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:09.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:09.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:10.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:10.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:10.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:10.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:11.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:11.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:11.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:11.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:12.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:12.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:12.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:12.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:13.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:13.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:13.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:13.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:14.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:14.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:14.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:14.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:15.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:15.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:15.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:15.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:16.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:16.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:16.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:16.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:17.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:17.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:17.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:17.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:18.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:18.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:18.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:18.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:19.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:19.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:19.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:19.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:20.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:20.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:20.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:20.260+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:21.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:21.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:21.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:21.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:22.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:22.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:22.274 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:22.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:23.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:23.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:23.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:23.305+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:24.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:24.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:24.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:24.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:25.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:25.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:25.252 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:25.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:26.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:26.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:26.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:26.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:27.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:27.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:27.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:27.241+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:28.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:28.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:28.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:28.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:29.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:29.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:29.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:29.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:30.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:30.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:30.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:30.349+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:31.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:31.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:31.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:31.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:32.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:32.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:32.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:32.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:33.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:33.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:33.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:33.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:34.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:34.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:34.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:34.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:35.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:35.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:35.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:35.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:36.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:36.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:36.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:36.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:37.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:37.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:37.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:37.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:38.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:38.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:38.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:38.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:39.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:39.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:39.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:39.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:40.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:40.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:40.240 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:40.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:41.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:41.195+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:41.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:41.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:42.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:42.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:42.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:42.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:43.196 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:43.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:43.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:43.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:44.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:44.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:44.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:44.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:45.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:45.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:45.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:45.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:46.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:46.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:46.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:46.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:47.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:47.203+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:47.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:47.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:48.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:48.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:48.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:48.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:49.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:49.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:49.408 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:49.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:50.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:50.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:50.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:50.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:51.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:51.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:51.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:51.402+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:52.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:52.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:52.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:52.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:53.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:53.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:53.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:53.385+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:54.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:54.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:54.361 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:54.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:55.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:55.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:55.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:55.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:56.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:56.090+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:56.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:56.299+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:57.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:57.099+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:57.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:57.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:58.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:58.111+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:58.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:58.324+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:59.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:41:59.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:41:59.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:41:59.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:00.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:00.064+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:00.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:00.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:01.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:01.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:01.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:01.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:02.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:02.050+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:02.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:02.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:03.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:03.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:03.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:03.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:04.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:04.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:04.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:04.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:05.063 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:05.063+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:05.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:05.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:06.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:06.091+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:06.356 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:06.356+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:07.080 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:07.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:07.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:07.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:08.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:08.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:08.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:08.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:09.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:09.083+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:09.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:09.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:10.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:10.109+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:10.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:10.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:11.130 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:11.129+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:11.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:11.429+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:12.123 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:12.122+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:12.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:12.429+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:13.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:13.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:13.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:13.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:14.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:14.097+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:14.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:14.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:15.084 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:15.083+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:15.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:15.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:16.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:16.039+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:16.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:16.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:17.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:17.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:17.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:17.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:18.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:18.039+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:18.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:18.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:19.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:19.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:19.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:19.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:20.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:20.062+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:20.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:20.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:21.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:21.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:21.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:21.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:22.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:22.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:22.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:22.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:23.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:23.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:23.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:23.687+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:24.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:24.075+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:24.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:24.695+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:25.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:25.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:25.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:25.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:26.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:26.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:26.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:26.720+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:27.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:27.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:27.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:27.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:28.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:28.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:28.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:28.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:29.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:29.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:29.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:29.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:30.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:30.147+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:30.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:30.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:31.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:31.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:31.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:31.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:32.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:32.187+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:32.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:32.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:33.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:33.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:33.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:33.638+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:34.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:34.242+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:34.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:34.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:35.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:35.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:35.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:35.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:36.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:36.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:36.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:36.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:37.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:37.205+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:37.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:37.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:38.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:38.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:38.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:38.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:39.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:39.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:39.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:39.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:40.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:40.222+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:40.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:40.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:41.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:41.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:41.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:41.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:42.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:42.188+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:42.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:42.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:43.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:43.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:43.803 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:43.803+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:44.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:44.210+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:44.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:44.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:45.211 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:45.211+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:45.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:45.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:46.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:46.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:46.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:46.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:47.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:47.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:47.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:47.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:48.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:48.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:48.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:48.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:49.146 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:49.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:49.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:49.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:50.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:50.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:50.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:50.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:51.146 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:51.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:51.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:51.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:52.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:52.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:52.754 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:52.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:53.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:53.222+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:53.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:53.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:54.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:54.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:54.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:54.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:55.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:55.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:55.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:55.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:56.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:56.179+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:56.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:56.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:57.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:57.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:57.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:57.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:58.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:58.138+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:58.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:58.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:59.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:42:59.108+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:42:59.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:42:59.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:00.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:00.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:00.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:00.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:01.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:01.113+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:01.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:01.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:02.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:02.089+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:02.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:02.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:03.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:03.109+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:03.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:03.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:04.128 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:04.128+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:04.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:04.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:05.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:05.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:05.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:05.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:06.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:06.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:06.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:06.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:07.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:07.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:07.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:07.639+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:08.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:08.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:08.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:08.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:09.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:09.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:09.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:09.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:10.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:10.218+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:10.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:10.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:11.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:11.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:11.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:11.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:12.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:12.206+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:12.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:12.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:13.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:13.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:13.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:13.713+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:14.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:14.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:14.742 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:14.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:15.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:15.215+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:15.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:15.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:16.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:16.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:16.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:16.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:17.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:17.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:17.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:17.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:18.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:18.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:18.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:18.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:19.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:19.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:19.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:19.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:20.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:20.228+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:20.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:20.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:21.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:21.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:21.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:21.641+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:22.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:22.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:22.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:22.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:23.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:23.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:23.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:23.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:24.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:24.228+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:24.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:24.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:25.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:25.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:25.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:25.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:26.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:26.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:26.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:26.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:27.248 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:27.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:27.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:27.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:28.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:28.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:28.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:28.678+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:29.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:29.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:29.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:29.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:30.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:30.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:30.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:30.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:31.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:31.165+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:31.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:31.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:32.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:32.201+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:32.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:32.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:33.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:33.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:33.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:33.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:34.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:34.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:34.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:34.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:35.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:35.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:35.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:35.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:36.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:36.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:36.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:36.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:37.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:37.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:37.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:37.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:38.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:38.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:38.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:38.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:39.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:39.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:39.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:39.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:40.146 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:40.145+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:40.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:40.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:41.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:41.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:41.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:41.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:42.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:42.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:42.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:42.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:43.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:43.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:43.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:43.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:44.310 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:44.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:44.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:44.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:45.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:45.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:45.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:45.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:46.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:46.284+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:46.451 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:46.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:47.247 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:47.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:47.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:47.493+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:48.283 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:48.283+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:48.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:48.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:49.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:49.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:49.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:49.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:50.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:50.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:50.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:50.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:51.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:51.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:51.501 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:51.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:52.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:52.188+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:52.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:52.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:53.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:53.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:53.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:53.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:54.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:54.165+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:54.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:54.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:55.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:55.181+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:55.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:55.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:56.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:56.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:56.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:56.542+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:57.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:57.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:57.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:57.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:58.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:58.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:58.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:58.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:59.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:43:59.111+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:43:59.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:43:59.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:00.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:00.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:00.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:00.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:01.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:01.143+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:01.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:01.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:02.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:02.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:02.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:02.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:03.210 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:03.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:03.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:03.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:04.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:04.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:04.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:04.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:05.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:05.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:05.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:05.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:06.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:06.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:06.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:06.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:07.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:07.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:07.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:07.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:08.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:08.230+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:08.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:08.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:09.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:09.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:09.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:09.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:10.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:10.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:10.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:10.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:11.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:11.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:11.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:11.663+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:12.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:12.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:12.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:12.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:13.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:13.188+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:13.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:13.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:14.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:14.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:14.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:14.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:15.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:15.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:15.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:15.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:16.103 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:16.102+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:16.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:16.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:17.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:17.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:17.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:17.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:18.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:18.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:18.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:18.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:19.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:19.113+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:19.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:19.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:20.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:20.120+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:20.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:20.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:21.165 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:21.166+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:21.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:21.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:22.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:22.122+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:22.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:22.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:23.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:23.149+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:23.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:23.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:24.179 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:24.179+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:24.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:24.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:25.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:25.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:25.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:25.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:26.085 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:26.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:26.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:26.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:27.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:27.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:27.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:27.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:28.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:28.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:28.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:28.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:29.099 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:29.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:29.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:29.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:30.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:30.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:30.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:30.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:31.074 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:31.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:31.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:31.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:32.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:32.050+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:32.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:32.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:33.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:33.083+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:33.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:33.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:34.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:34.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:34.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:34.718+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:35.125 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:35.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:35.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:35.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:36.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:36.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:36.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:36.701+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:37.173 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:37.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:37.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:37.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:38.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:38.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:38.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:38.701+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:39.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:39.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:39.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:39.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:40.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:40.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:40.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:40.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:41.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:41.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:41.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:41.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:42.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:42.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:42.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:42.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:43.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:43.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:43.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:43.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:44.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:44.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:44.659 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:44.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:45.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:45.279+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:45.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:45.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:46.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:46.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:46.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:46.667+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:47.279 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:47.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:47.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:47.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:48.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:48.305+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:48.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:48.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:49.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:49.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:49.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:49.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:50.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:50.386+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:50.742 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:50.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:51.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:51.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:51.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:51.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:52.348 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:52.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:52.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:52.741+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:53.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:53.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:53.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:53.766+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:54.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:54.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:54.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:54.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:55.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:55.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:55.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:55.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:56.299 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:56.299+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:56.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:56.775+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:57.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:57.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:57.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:57.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:58.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:58.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:58.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:58.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:59.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:44:59.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:44:59.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:44:59.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:00.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:00.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:00.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:00.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:01.367 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:01.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:01.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:01.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:02.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:02.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:02.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:02.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:03.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:03.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:03.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:03.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:04.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:04.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:04.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:04.895+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:05.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:05.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:05.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:05.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:06.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:06.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:06.909 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:06.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:07.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:07.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:07.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:07.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:08.228 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T20:45:08.228+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x7ffb6400a020 msgr2=0x7ffb6c046e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T20:45:08.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:08.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:08.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:08.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:09.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:09.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:09.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:09.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:10.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:10.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:11.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:11.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:11.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:11.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:12.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:12.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:12.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:12.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:13.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:13.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:13.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:13.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:14.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:14.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:14.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:14.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:14.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:14.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:15.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:15.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:16.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:16.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:16.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:16.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:16.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:16.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:17.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:17.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:18.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:18.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:18.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:18.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:18.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:18.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:19.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:19.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:19.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:19.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:20.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:20.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:20.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:20.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:21.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:21.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:21.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:21.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:22.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:22.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:22.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:22.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:23.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:23.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:24.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:24.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:24.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:24.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:25.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:25.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:25.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:25.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:26.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:26.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:26.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:26.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:26.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:26.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:27.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:27.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:27.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:27.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:28.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:28.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:28.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:28.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:29.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:29.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:29.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:29.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:30.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:30.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:30.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:30.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:31.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:31.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:32.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:32.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:32.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:32.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:33.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:33.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:33.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:33.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:33.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:33.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:34.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:34.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:34.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:34.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:35.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:35.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:35.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:35.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:36.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:36.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:37.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:36.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:37.671 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:37.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:37.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:37.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:38.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:38.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:39.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:39.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:39.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:39.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:39.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:39.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:40.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:40.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:40.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:40.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:41.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:41.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:41.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:41.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:42.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:42.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:42.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:42.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:43.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:43.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:43.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:43.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:44.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:44.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:44.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:44.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:45.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:45.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:45.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:45.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:46.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:46.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:46.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:46.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:47.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:47.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:47.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:47.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:48.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:48.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:48.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:48.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:49.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:49.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:49.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:49.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:50.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:50.736+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:50.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:50.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:51.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:51.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:51.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:51.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:52.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:52.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:53.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:53.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:53.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:53.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:54.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:54.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:54.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:54.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:55.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:55.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:55.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:55.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:56.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:56.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:56.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:56.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:57.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:57.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:57.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:57.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:58.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:58.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:58.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:58.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:59.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:45:59.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:45:59.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:45:59.672+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:00.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:00.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:00.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:00.626+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:01.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:01.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:01.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:01.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:02.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:02.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:02.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:02.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:03.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:03.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:03.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:03.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:04.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:04.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:04.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:04.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:05.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:05.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:05.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:05.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:06.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:06.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:06.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:06.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:07.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:07.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:07.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:07.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:08.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:08.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:08.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:08.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:09.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:09.282+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:09.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:09.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:10.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:10.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:10.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:10.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:11.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:11.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:11.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:11.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:12.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:12.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:12.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:12.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:13.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:13.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:13.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:13.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:14.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:14.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:14.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:14.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:15.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:15.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:15.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:15.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:16.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:16.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:16.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:16.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:17.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:17.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:17.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:17.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:18.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:18.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:18.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:18.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:19.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:19.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:19.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:19.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:20.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:20.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:20.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:20.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:21.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:21.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:21.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:21.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:22.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:22.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:22.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:22.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:23.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:23.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:23.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:23.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:24.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:24.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:24.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:24.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:25.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:25.316+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:25.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:25.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:26.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:26.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:26.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:26.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:27.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:27.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:27.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:27.623+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:28.248 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:28.248+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:28.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:28.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:29.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:29.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:29.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:29.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:30.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:30.196+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:30.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:30.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:31.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:31.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:31.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:31.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:32.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:32.246+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:32.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:32.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:33.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:33.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:33.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:33.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:34.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:34.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:34.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:34.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:35.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:35.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:35.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:35.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:36.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:36.327+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:36.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:36.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:37.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:37.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:37.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:37.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:38.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:38.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:38.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:38.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:39.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:39.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:39.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:39.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:40.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:40.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:40.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:40.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:41.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:41.312+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:41.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:41.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:42.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:42.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:42.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:42.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:43.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:43.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:43.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:43.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:44.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:44.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:44.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:44.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:45.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:45.357+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:45.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:45.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:46.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:46.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:46.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:46.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:47.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:47.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:47.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:47.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:48.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:48.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:48.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:48.680+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:49.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:49.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:49.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:49.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:50.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:50.340+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:50.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:50.657+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:51.369 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:51.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:51.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:51.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:52.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:52.412+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:52.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:52.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:53.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:53.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:53.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:53.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:54.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:54.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:54.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:54.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:55.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:55.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:55.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:55.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:56.375 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:56.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:56.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:56.580+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:57.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:57.419+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:57.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:57.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:58.398 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:58.398+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:58.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:58.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:59.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:46:59.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:46:59.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:46:59.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:00.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:00.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:00.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:00.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:01.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:01.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:01.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:01.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:02.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:02.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:02.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:02.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:03.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:03.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:03.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:03.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:04.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:04.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:04.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:04.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:05.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:05.378+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:05.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:05.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:06.385 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:06.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:06.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:06.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:07.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:07.429+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:07.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:07.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:08.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:08.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:08.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:08.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:09.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:09.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:09.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:09.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:10.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:10.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:10.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:10.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:11.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:11.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:11.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:11.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:12.406 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:12.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:12.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:12.444+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:13.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:13.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:13.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:13.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:14.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:14.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:15.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:14.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:15.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:15.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:15.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:15.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:16.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:16.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:17.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:16.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:17.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:17.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:18.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:17.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:18.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:18.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:18.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:18.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:19.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:19.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:19.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:19.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:20.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:20.400+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:20.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:20.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:21.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:21.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:21.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:21.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:22.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:22.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:22.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:22.415+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:23.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:23.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:23.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:23.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:24.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:24.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:24.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:24.404+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:25.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:25.357+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:25.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:25.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:26.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:26.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:26.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:26.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:27.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:27.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:27.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:27.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:28.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:28.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:28.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:28.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:29.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:29.357+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:29.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:29.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:30.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:30.347+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:30.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:30.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:31.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:31.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:31.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:31.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:32.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:32.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:32.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:32.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:33.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:33.397+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:33.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:33.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:34.391 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:34.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:34.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:34.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:35.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:35.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:35.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:35.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:36.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:36.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:36.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:36.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:37.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:37.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:37.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:37.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:38.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:38.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:38.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:38.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:39.406 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:39.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:39.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:39.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:40.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:40.420+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:40.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:40.539+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:41.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:41.417+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:41.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:41.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:42.395 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:42.395+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:42.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:42.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:43.413 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:43.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:43.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:43.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:44.422 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:44.422+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:44.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:44.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:45.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:45.451+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:45.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:45.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:46.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:46.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:46.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:46.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:47.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:47.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:47.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:47.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:48.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:48.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:48.508 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:48.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:49.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:49.446+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:49.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:49.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:50.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:50.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:50.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:50.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:51.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:51.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:51.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:51.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:52.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:52.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:52.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:52.504+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:53.456 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:53.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:53.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:53.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:54.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:54.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:54.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:54.496+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:55.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:55.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:55.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:55.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:56.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:56.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:56.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:56.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:57.443 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:57.443+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:57.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:57.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:58.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:58.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:58.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:58.478+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:59.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:47:59.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:47:59.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:47:59.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:00.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:00.510+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:00.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:00.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:01.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:01.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:01.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:01.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:02.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:02.510+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:02.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:02.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:03.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:03.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:03.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:03.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:04.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:04.467+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:04.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:04.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:05.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:05.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:05.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:05.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:06.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:06.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:06.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:06.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:07.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:07.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:07.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:07.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:08.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:08.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:08.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:08.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:09.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:09.516+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:09.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:09.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:10.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:10.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:10.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:10.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:11.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:11.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:11.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:11.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:12.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:12.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:12.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:12.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:13.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:13.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:14.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:13.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:14.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:14.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:14.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:14.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:15.481 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:15.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:15.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:15.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:16.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:16.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:16.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:16.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:17.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:17.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:17.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:17.624+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:18.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:18.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:18.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:18.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:19.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:19.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:19.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:19.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:20.467 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:20.467+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:20.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:20.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:20.884 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T20:48:20.885+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589432a6880 msgr2=0x7ffb6c05fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-09-29T20:48:21.450 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:21.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:21.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:21.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:22.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:22.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:22.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:22.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:23.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:23.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:23.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:23.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:24.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:24.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:24.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:24.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:25.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:25.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:25.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:25.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:26.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:26.499+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:26.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:26.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:27.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:27.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:27.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:27.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:28.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:28.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:28.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:28.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:29.480 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:29.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:29.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:29.651+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:30.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:30.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:32.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:30.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:32.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:31.480+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:32.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:31.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:32.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:32.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:32.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:32.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:33.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:33.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:33.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:33.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:34.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:34.504+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:34.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:34.569+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:35.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:35.534+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:35.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:35.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:36.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:36.490+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:36.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:36.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:37.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:37.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:37.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:37.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:38.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:38.449+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:38.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:38.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:39.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:39.405+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:40.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:39.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:40.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:40.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:40.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:40.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:41.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:41.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:41.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:41.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:42.486 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:42.487+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:42.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:42.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:43.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:43.485+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:43.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:43.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:44.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:44.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:44.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:44.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:45.448 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:45.448+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:45.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:45.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:46.445 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:46.446+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:46.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:46.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:47.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:47.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:47.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:47.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:48.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:48.435+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:48.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:48.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:49.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:49.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:49.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:49.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:50.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:50.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:50.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:50.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:51.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:51.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:51.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:51.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:52.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:52.340+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:52.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:52.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:53.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:53.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:53.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:53.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:54.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:54.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:54.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:54.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:55.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:55.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:55.615 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:55.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:56.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:56.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:56.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:56.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:57.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:57.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:57.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:57.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:58.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:58.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:58.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:58.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:59.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:48:59.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:48:59.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:48:59.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:00.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:00.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:00.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:00.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:01.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:01.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:01.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:01.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:02.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:02.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:02.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:02.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:03.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:03.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:03.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:03.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:04.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:04.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:04.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:04.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:05.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:05.299+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:05.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:05.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:06.335 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:06.335+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:06.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:06.514+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:07.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:07.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:07.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:07.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:08.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:08.288+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:08.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:08.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:09.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:09.302+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:09.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:09.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:10.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:10.308+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:10.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:10.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:11.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:11.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:11.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:11.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:12.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:12.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:12.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:12.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:13.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:13.259+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:13.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:13.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:14.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:14.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:14.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:14.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:15.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:15.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:15.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:15.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:16.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:16.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:16.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:16.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:17.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:17.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:17.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:17.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:18.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:18.277+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:18.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:18.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:19.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:19.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:19.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:19.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:20.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:20.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:20.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:20.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:21.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:21.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:21.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:21.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:22.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:22.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:22.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:22.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:23.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:23.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:23.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:23.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:24.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:24.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:24.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:24.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:25.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:25.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:25.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:25.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:26.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:26.277+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:26.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:26.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:27.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:27.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:27.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:27.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:28.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:28.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:28.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:28.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:29.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:29.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:29.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:29.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:30.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:30.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:30.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:30.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:31.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:31.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:31.616 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:31.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:32.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:32.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:32.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:32.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:33.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:33.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:33.643 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:33.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:34.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:34.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:34.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:34.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:35.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:35.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:35.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:35.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:36.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:36.199+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:36.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:36.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:37.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:37.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:37.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:37.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:38.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:38.269+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:38.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:38.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:39.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:39.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:39.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:39.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:40.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:40.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:40.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:40.717+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:41.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:41.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:41.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:41.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:42.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:42.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:42.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:42.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:43.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:43.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:43.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:43.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:44.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:44.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:44.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:44.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:45.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:45.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:45.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:45.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:46.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:46.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:46.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:46.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:47.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:47.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:47.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:47.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:48.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:48.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:48.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:48.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:49.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:49.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:49.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:49.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:50.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:50.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:50.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:50.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:51.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:51.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:51.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:51.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:52.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:52.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:52.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:52.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:53.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:53.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:53.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:53.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:53.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:53.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:54.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:54.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:55.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:55.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:55.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:55.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:56.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:56.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:56.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:56.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:57.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:57.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:57.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:57.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:58.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:58.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:58.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:58.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:58.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:49:58.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:49:59.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:49:59.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:00.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:00.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:00.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:00.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:01.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:01.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:01.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:01.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:02.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:02.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:02.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:02.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:03.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:03.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:03.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:03.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:04.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:04.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:04.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:04.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:05.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:05.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:05.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:05.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:06.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:06.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:06.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:06.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:07.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:07.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:07.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:07.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:08.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:08.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:08.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:08.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:09.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:09.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:09.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:09.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:10.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:10.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:10.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:10.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:11.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:11.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:11.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:11.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:12.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:12.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:12.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:12.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:13.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:13.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:13.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:13.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:14.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:14.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:14.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:14.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:15.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:15.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:15.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:15.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:16.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:16.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:16.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:16.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:17.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:17.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:17.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:17.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:18.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:18.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:18.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:18.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:19.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:19.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:19.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:19.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:20.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:20.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:20.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:20.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:21.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:21.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:21.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:21.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:22.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:22.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:22.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:22.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:23.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:23.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:23.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:23.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:24.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:24.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:24.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:24.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:25.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:25.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:25.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:25.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:26.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:26.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:26.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:26.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:27.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:27.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:27.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:27.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:28.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:28.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:28.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:28.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:29.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:29.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:29.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:29.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:30.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:30.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:30.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:30.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:31.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:31.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:31.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:31.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:32.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:32.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:32.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:32.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:33.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:33.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:33.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:33.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:34.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:34.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:34.758 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:34.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:35.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:35.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:35.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:35.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:36.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:36.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:36.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:36.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:36.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:36.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:37.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:37.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:37.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:37.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:38.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:38.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:38.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:38.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:39.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:39.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:39.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:39.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:40.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:40.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:40.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:40.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:41.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:41.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:42.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:42.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:42.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:42.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:42.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:42.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:43.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:43.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:43.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:43.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:44.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:44.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:46.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:44.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:46.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:45.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:46.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:45.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:46.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:46.850+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:47.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:46.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:47.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:47.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:48.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:48.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:48.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:48.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:48.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:48.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:49.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:49.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:50.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:50.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:50.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:50.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:51.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:51.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:51.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:51.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:52.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:52.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:53.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:53.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:53.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:53.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:53.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:53.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:54.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:54.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:54.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:54.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:55.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:55.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:56.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:56.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:56.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:56.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:56.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:56.996+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:57.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:57.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:58.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:58.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:58.012 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:58.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:59.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:59.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:59.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:50:59.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:50:59.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:50:59.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:00.027 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:00.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:01.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:01.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:01.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:01.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:01.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:01.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:02.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:02.068+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:02.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:02.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:03.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:03.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:03.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:03.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:04.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:04.009+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:04.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:04.995+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:05.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:05.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:06.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:06.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:06.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:06.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:07.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:07.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:07.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:07.050+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:08.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:08.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:08.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:08.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:09.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:09.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:09.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:09.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:10.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:10.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:10.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:10.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:10.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:10.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:11.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:11.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:11.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:11.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:12.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:12.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:13.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:13.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:13.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:13.116+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:14.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:14.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:14.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:14.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:14.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:14.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:15.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:15.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:16.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:16.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:16.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:16.158+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:17.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:17.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:17.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:17.165+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:17.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:17.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:18.146 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:18.146+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:19.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:19.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:19.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:19.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:20.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:20.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:20.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:20.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:21.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:21.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:21.161 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:21.160+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:22.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:22.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:22.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:22.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:23.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:23.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:23.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:23.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:24.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:24.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:24.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:24.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:25.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:25.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:25.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:25.252+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:26.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:26.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:26.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:26.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:27.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:27.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:27.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:27.278+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:28.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:28.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:28.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:28.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:29.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:29.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:29.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:29.256+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:30.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:30.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:30.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:30.260+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:31.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:31.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:31.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:31.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:32.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:32.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:32.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:32.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:33.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:33.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:33.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:33.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:34.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:34.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:34.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:34.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:35.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:35.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:35.260 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:35.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:36.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:36.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:36.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:36.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:37.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:37.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:37.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:37.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:38.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:38.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:38.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:38.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:39.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:39.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:39.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:39.307+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:39.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:39.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:40.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:40.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:40.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:40.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:41.318 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:41.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:41.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:41.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:42.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:42.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:42.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:42.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:43.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:43.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:43.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:43.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:44.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:44.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:44.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:44.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:45.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:45.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:45.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:45.876+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:46.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:46.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:46.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:46.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:47.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:47.224+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:47.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:47.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:48.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:48.208+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:48.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:48.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:49.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:49.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:49.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:49.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:50.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:50.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:50.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:50.912+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:51.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:51.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:51.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:51.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:52.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:52.174+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:52.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:52.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:53.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:53.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:53.977 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:53.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:54.186 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:54.186+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:54.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:54.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:55.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:55.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:55.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:55.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:56.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:56.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:56.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:56.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:57.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:57.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:57.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:57.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:58.182 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:58.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:58.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:58.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:59.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:51:59.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:51:59.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:51:59.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:00.132 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:00.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:00.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:00.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:01.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:01.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:01.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:01.912+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:02.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:02.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:02.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:02.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:03.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:03.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:03.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:03.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:04.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:04.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:05.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:05.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:05.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:05.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:06.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:06.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:06.061 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:06.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:07.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:07.044+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:07.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:07.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:08.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:08.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:08.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:08.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:09.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:09.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:09.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:09.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:09.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:10.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:10.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:10.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:11.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:11.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:11.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:11.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:12.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:12.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:12.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:12.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:13.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:13.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:13.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:13.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:14.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:14.068+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:14.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:14.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:15.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:15.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:15.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:15.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:16.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:16.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:16.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:16.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:17.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:17.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:17.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:17.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:18.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:18.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:18.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:18.046+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:19.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:19.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:19.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:19.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:20.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:20.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:20.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:20.055+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:21.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:21.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:21.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:21.094+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:22.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:22.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:22.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:22.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:23.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:23.046+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:23.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:23.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:24.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:24.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:24.040 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:24.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:24.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:24.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:25.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:25.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:26.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:26.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:26.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:26.090+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:27.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:27.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:27.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:27.127+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:27.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:27.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:28.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:28.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:28.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:28.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:29.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:29.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:29.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:29.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:30.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:30.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:30.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:30.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:31.063 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:31.063+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:32.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:32.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:32.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:32.082+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:33.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:33.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:33.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:33.079+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:34.060 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:34.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:34.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:34.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:35.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:35.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:35.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:35.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:35.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:35.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:36.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:36.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:37.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:37.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:37.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:37.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:37.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:37.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:38.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:38.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:38.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:38.945+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:39.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:39.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:39.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:39.979+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:40.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:40.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:40.967 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:40.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:41.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:41.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:41.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:41.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:42.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:42.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:42.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:42.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:43.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:43.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:43.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:43.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:44.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:44.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:44.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:44.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:45.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:45.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:45.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:45.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:46.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:46.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:46.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:46.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:47.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:47.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:47.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:47.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:48.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:48.048+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:48.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:48.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:49.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:49.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:49.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:49.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:50.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:50.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:50.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:50.854+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:51.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:51.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:51.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:51.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:52.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:52.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:52.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:52.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:53.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:53.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:53.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:53.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:54.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:54.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:54.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:54.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:55.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:55.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:55.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:55.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:56.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:56.024+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:56.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:56.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:57.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:57.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:57.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:57.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:58.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:58.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:58.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:58.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:59.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:52:59.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:52:59.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:52:59.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:00.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:00.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:00.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:00.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:01.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:01.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:01.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:01.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:02.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:02.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:02.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:02.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:03.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:03.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:03.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:03.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:04.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:04.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:04.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:04.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:05.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:05.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:05.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:05.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:06.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:06.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:06.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:06.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:07.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:07.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:07.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:07.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:08.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:08.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:08.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:08.793+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:09.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:09.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:09.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:09.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:10.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:10.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:10.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:10.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:11.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:11.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:11.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:11.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:12.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:12.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:12.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:12.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:13.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:13.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:13.766 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:13.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:14.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:14.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:14.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:14.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:15.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:15.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:15.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:15.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:16.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:16.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:16.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:16.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:16.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:16.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:17.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:17.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:18.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:18.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:18.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:18.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:18.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:18.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:19.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:19.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:20.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:20.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:20.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:20.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:21.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:21.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:21.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:21.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:22.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:22.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:22.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:22.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:23.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:23.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:23.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:23.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:24.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:24.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:24.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:24.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:25.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:25.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:25.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:25.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:26.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:26.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:26.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:26.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:26.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:26.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:27.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:27.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:27.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:27.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:28.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:28.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:29.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:29.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:29.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:29.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:30.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:30.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:30.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:30.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:31.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:31.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:31.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:31.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:31.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:31.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:32.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:32.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:33.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:33.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:33.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:33.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:34.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:34.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:34.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:34.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:35.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:35.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:35.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:35.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:36.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:36.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:36.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:36.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:37.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:37.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:37.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:37.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:38.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:38.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:38.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:38.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:39.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:39.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:39.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:39.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:40.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:40.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:40.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:40.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:41.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:41.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:41.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:41.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:42.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:42.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:42.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:42.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:43.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:43.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:43.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:43.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:44.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:44.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:44.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:44.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:45.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:45.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:45.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:45.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:46.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:46.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:46.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:46.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:47.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:47.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:47.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:47.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:48.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:48.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:48.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:48.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:49.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:49.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:49.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:49.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:50.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:50.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:50.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:50.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:51.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:51.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:51.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:51.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:52.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:52.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:52.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:52.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:53.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:53.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:53.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:53.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:54.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:54.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:54.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:54.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:55.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:55.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:55.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:55.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:56.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:56.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:56.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:56.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:57.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:57.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:57.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:57.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:58.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:58.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:58.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:58.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:59.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:53:59.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:53:59.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:53:59.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:00.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:00.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:00.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:00.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:01.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:01.239+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:01.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:01.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:02.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:02.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:02.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:02.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:03.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:03.182+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:03.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:03.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:04.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:04.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:04.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:04.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:05.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:05.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:05.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:05.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:06.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:06.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:06.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:06.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:07.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:07.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:07.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:07.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:08.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:08.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:08.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:08.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:09.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:09.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:09.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:09.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:10.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:10.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:10.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:10.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:11.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:11.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:11.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:11.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:12.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:12.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:12.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:12.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:13.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:13.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:13.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:13.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:14.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:14.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:14.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:14.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:15.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:15.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:15.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:15.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:16.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:16.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:16.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:16.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:17.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:17.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:17.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:17.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:18.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:18.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:18.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:18.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:19.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:19.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:19.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:19.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:20.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:20.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:20.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:20.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:21.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:21.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:21.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:21.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:22.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:22.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:22.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:22.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:23.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:23.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:23.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:23.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:24.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:24.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:24.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:24.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:25.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:25.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:25.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:25.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:26.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:26.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:26.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:26.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:27.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:27.224+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:27.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:27.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:28.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:28.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:28.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:28.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:29.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:29.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:29.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:29.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:30.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:30.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:30.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:30.751+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:31.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:31.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:31.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:31.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:32.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:32.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:32.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:32.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:33.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:33.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:33.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:33.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:34.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:34.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:34.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:34.743+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:35.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:35.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:35.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:35.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:36.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:36.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:36.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:36.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:37.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:37.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:37.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:37.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:38.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:38.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:38.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:38.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:39.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:39.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:39.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:39.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:40.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:40.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:40.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:40.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:41.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:41.123+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:41.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:41.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:42.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:42.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:42.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:42.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:43.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:43.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:43.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:43.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:44.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:44.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:44.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:44.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:45.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:45.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:45.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:45.569+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:46.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:46.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:46.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:46.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:47.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:47.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:47.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:47.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:48.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:48.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:48.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:48.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:49.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:49.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:49.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:49.551+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:50.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:50.199+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:50.575 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:50.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:51.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:51.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:51.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:51.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:52.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:52.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:52.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:52.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:53.145 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:53.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:53.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:53.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:54.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:54.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:54.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:54.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:55.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:55.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:55.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:55.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:56.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:56.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:56.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:56.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:57.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:57.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:57.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:57.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:58.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:58.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:58.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:58.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:59.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:54:59.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:54:59.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:54:59.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:00.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:00.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:00.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:00.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:01.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:01.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:01.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:01.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:02.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:02.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:02.717 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:02.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:03.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:03.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:03.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:03.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:04.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:04.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:04.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:04.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:05.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:05.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:05.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:05.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:06.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:06.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:06.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:06.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:07.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:07.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:07.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:07.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:08.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:08.048+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:08.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:08.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:09.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:09.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:09.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:09.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:09.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:09.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:10.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:10.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:10.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:10.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:11.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:11.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:11.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:11.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:12.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:12.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:12.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:12.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:13.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:13.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:13.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:13.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:14.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:14.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:14.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:14.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:15.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:15.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:15.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:15.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:16.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:16.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:16.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:16.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:17.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:17.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:17.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:17.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:18.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:18.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:18.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:18.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:19.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:19.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:19.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:19.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:20.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:20.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:20.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:20.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:21.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:21.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:21.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:21.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:22.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:22.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:22.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:22.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:23.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:23.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:23.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:23.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:24.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:24.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:24.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:24.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:25.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:25.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:25.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:25.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:26.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:26.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:26.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:26.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:27.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:27.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:27.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:27.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:28.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:28.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:28.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:28.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:29.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:29.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:29.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:29.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:30.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:30.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:30.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:30.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:31.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:31.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:31.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:31.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:32.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:32.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:32.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:32.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:33.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:33.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:33.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:33.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:34.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:34.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:34.869 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:34.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:35.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:35.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:35.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:35.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:36.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:36.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:36.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:36.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:37.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:37.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:37.783 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:37.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:38.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:38.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:38.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:38.800+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:39.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:39.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:39.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:39.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:40.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:40.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:40.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:40.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:41.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:41.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:41.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:41.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:42.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:42.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:42.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:42.790+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:43.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:43.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:43.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:43.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:44.498 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:44.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:44.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:44.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:45.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:45.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:45.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:45.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:46.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:46.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:46.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:46.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:47.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:47.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:47.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:47.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:48.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:48.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:48.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:48.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:49.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:49.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:49.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:49.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:50.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:50.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:50.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:50.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:51.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:51.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:51.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:51.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:52.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:52.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:52.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:52.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:53.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:53.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:53.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:53.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:54.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:54.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:54.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:54.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:55.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:55.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:55.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:55.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:56.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:56.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:56.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:56.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:57.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:57.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:57.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:57.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:58.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:58.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:59.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:55:59.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:55:59.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:55:59.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:00.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:00.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:00.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:00.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:01.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:01.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:01.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:01.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:02.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:02.001+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:02.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:02.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:03.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:03.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:03.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:03.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:04.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:04.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:04.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:04.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:04.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:04.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:05.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:05.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:06.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:06.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:06.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:06.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:06.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:06.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:07.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:07.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:08.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:08.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:08.673 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:08.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:08.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:08.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:09.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:09.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:09.971 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:09.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:10.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:10.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:10.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:10.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:11.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:11.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:11.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:11.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:12.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:12.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:12.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:12.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:13.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:13.610+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:13.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:13.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:14.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:14.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:14.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:14.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:15.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:15.566+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:15.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:15.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:16.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:16.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:16.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:16.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:17.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:17.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:17.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:17.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:18.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:18.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:18.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:18.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:19.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:19.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:19.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:19.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:20.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:20.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:20.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:20.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:21.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:21.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:21.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:21.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:22.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:22.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:22.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:22.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:23.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:23.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:24.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:24.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:24.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:24.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:25.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:25.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:25.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:25.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:26.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:26.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:26.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:26.615+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:27.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:27.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:27.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:27.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:28.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:28.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:28.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:28.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:29.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:29.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:29.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:29.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:30.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:30.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:30.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:30.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:31.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:31.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:31.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:31.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:32.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:32.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:32.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:32.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:33.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:33.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:33.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:33.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:34.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:34.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:34.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:34.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:35.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:35.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:35.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:35.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:36.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:36.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:36.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:36.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:37.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:37.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:37.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:37.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:38.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:38.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:38.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:38.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:39.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:39.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:39.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:39.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:40.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:40.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:40.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:40.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:41.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:41.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:41.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:41.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:42.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:42.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:42.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:42.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:43.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:43.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:43.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:43.478+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:44.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:44.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:44.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:44.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:45.259 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:45.259+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:45.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:45.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:46.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:46.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:46.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:46.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:47.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:47.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:47.579 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:47.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:48.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:48.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:48.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:48.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:49.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:49.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:49.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:49.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:50.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:50.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:50.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:50.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:51.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:51.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:51.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:51.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:52.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:52.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:52.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:52.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:53.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:53.048+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:53.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:53.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:54.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:54.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:54.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:54.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:54.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:54.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:55.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:55.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:55.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:55.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:56.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:56.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:56.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:56.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:57.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:57.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:57.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:57.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:58.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:58.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:58.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:58.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:59.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:56:59.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:56:59.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:56:59.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:00.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:00.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:00.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:00.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:01.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:01.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:01.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:01.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:02.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:02.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:02.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:02.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:03.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:03.554+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:03.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:03.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:04.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:04.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:05.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:05.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:05.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:05.599+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:06.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:06.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:06.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:06.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:07.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:07.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:07.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:07.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:08.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:08.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:08.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:08.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:09.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:09.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:09.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:09.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:10.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:10.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:10.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:10.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:11.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:11.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:11.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:11.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:12.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:12.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:12.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:12.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:13.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:13.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:13.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:13.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:14.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:14.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:14.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:14.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:15.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:15.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:15.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:15.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:16.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:16.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:16.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:16.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:17.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:17.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:17.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:17.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:18.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:18.123+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:18.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:18.602+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:19.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:19.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:19.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:19.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:20.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:20.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:20.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:20.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:21.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:21.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:21.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:21.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:22.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:22.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:22.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:22.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:23.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:23.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:23.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:23.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:24.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:24.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:24.614 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:24.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:25.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:25.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:25.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:25.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:26.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:26.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:26.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:26.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:27.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:27.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:27.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:27.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:28.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:28.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:28.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:28.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:29.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:29.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:29.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:29.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:30.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:30.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:30.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:30.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:31.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:31.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:31.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:31.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:31.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:31.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:32.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:32.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:33.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:33.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:33.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:33.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:34.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:34.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:34.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:34.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:35.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:35.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:35.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:35.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:35.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:35.971+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:36.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:36.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:36.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:36.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:37.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:37.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:38.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:38.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:38.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:38.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:39.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:39.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:39.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:39.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:40.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:40.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:40.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:40.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:41.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:41.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:41.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:41.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:42.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:42.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:42.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:42.614+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:43.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:43.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:43.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:43.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:44.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:44.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:44.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:44.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:45.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:45.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:45.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:45.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:46.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:46.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:46.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:46.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:47.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:47.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:47.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:47.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:48.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:48.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:48.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:48.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:49.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:49.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:49.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:49.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:50.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:50.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:50.625 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:50.623+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:51.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:51.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:51.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:51.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:52.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:52.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:52.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:52.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:53.033 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:53.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:53.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:53.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:53.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:53.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:54.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:54.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:54.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:54.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:55.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:55.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:55.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:55.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:56.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:56.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:56.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:56.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:57.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:57.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:57.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:57.909+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:58.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:58.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:58.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:58.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:59.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:57:59.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:57:59.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:57:59.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:00.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:00.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:00.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:00.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:01.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:01.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:01.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:01.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:02.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:02.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:02.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:02.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:03.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:03.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:03.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:03.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:04.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:04.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:04.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:04.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:05.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:05.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:05.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:05.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:06.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:06.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:06.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:06.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:07.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:07.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:07.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:07.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:08.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:08.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:08.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:08.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:09.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:09.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:09.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:09.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:10.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:10.740+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:10.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:10.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:11.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:11.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:11.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:11.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:12.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:12.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:12.873 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:12.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:13.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:13.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:13.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:13.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:14.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:14.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:14.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:14.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:15.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:15.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:15.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:15.784+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:16.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:16.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:16.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:16.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:17.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:17.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:17.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:17.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:18.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:18.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:18.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:18.847+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:19.631 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:19.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:19.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:19.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:20.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:20.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:20.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:20.792+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:21.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:21.625+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:21.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:21.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:22.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:22.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:22.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:22.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:23.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:23.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:24.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:23.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:24.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:24.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:24.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:24.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:25.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:25.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:25.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:25.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:26.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:26.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:26.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:26.852+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:27.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:27.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:27.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:27.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:28.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:28.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:29.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:28.784+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:29.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:29.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:29.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:29.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:30.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:30.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:30.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:30.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:31.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:31.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:31.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:31.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:32.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:32.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:32.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:32.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:33.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:33.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:33.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:33.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:34.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:34.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:34.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:34.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:35.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:35.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:35.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:35.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:36.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:36.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:36.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:36.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:37.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:37.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:37.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:37.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:38.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:38.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:38.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:38.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:39.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:39.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:39.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:39.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:40.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:40.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:40.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:40.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:41.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:41.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:41.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:41.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:42.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:42.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:42.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:42.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:43.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:43.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:43.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:43.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:44.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:44.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:44.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:44.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:45.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:45.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:45.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:45.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:46.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:46.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:46.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:46.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:47.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:47.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:47.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:47.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:48.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:48.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:48.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:48.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:49.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:49.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:49.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:49.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:50.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:50.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:50.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:50.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:51.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:51.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:51.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:51.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:52.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:52.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:52.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:52.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:53.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:53.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:53.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:53.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:54.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:54.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:54.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:54.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:55.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:55.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:55.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:55.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:56.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:56.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:56.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:56.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:57.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:57.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:57.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:57.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:58.718 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:58.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:58.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:58.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:59.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:58:59.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:58:59.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:58:59.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:00.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:00.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:00.959 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:00.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:01.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:01.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:01.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:01.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:02.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:02.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:02.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:02.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:03.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:03.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:03.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:03.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:04.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:04.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:04.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:04.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:05.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:05.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:05.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:05.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:06.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:06.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:06.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:06.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:07.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:07.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:07.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:07.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:08.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:08.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:08.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:08.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:09.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:09.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:09.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:09.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:10.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:10.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:10.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:10.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:11.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:11.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:11.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:11.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:12.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:12.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:12.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:12.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:13.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:13.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:14.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:14.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:14.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:14.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:15.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:15.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:15.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:15.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:16.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:16.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:16.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:16.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:17.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:17.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:17.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:17.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:18.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:18.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:18.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:18.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:19.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:19.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:19.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:19.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:20.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:20.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:20.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:20.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:21.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:21.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:21.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:21.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:22.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:22.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:22.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:22.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:23.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:23.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:23.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:23.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:24.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:24.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:24.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:24.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:25.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:25.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:25.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:25.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:26.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:26.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:26.855 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:26.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:27.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:27.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:27.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:27.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:28.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:28.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:28.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:28.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:29.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:29.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:29.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:29.936+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:30.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:30.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:30.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:30.890+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:31.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:31.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:31.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:31.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:32.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:32.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:32.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:32.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:33.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:33.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:33.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:33.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:34.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:34.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:34.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:34.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:35.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:35.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:35.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:35.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:36.166 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:36.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:36.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:36.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:37.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:37.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:37.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:37.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:38.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:38.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:38.890 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:38.890+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:39.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:39.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:39.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:39.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:40.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:40.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:40.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:40.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:41.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:41.139+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:41.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:41.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:42.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:42.155+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:42.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:42.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:43.184 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:43.185+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:43.824 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:43.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:44.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:44.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:44.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:44.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:45.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:45.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:45.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:45.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:46.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:46.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:46.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:46.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:47.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:47.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:47.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:47.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:48.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:48.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:48.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:48.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:49.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:49.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:49.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:49.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:50.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:50.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:50.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:50.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:51.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:51.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:51.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:51.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:52.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:52.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:52.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:52.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:53.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:53.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:53.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:53.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:54.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:54.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:54.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:54.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:55.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:55.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:55.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:55.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:56.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:56.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:56.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:56.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:57.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:57.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:57.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:57.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:58.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:58.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:58.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:58.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:59.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T20:59:59.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T20:59:59.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T20:59:59.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:00.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:00.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:00.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:00.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:01.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:01.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:01.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:01.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:02.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:02.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:02.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:02.934+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:03.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:03.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:03.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:03.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:04.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:04.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:04.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:04.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:05.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:05.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:05.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:05.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:06.328 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:06.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:06.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:06.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:07.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:07.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:07.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:07.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:08.231 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T21:00:08.231+0000 7ffb849f5700 0 -- 172.21.15.143:0/2433915735 >> [v2:172.21.15.143:6808/86744,v1:172.21.15.143:6809/86744] conn(0x5589432a64b0 msgr2=0x7ffb6c0365b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-09-29T21:00:08.338 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:08.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:08.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:08.961+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:09.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:09.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:10.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:10.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:10.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:10.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:10.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:10.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:11.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:11.263+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:12.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:12.034+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:12.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:12.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:12.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:12.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:13.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:13.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:13.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:13.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:14.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:14.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:14.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:14.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:15.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:15.233+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:15.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:15.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:16.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:16.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:16.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:16.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:17.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:17.233+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:18.003 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:18.003+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:18.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:18.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:19.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:19.010+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:19.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:19.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:19.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:19.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:20.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:20.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:20.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:20.973+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:21.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:21.282+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:21.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:21.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:22.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:22.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:22.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:22.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:23.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:23.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:23.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:23.992+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:24.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:24.340+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:24.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:24.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:25.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:25.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:25.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:25.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:26.310 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:26.309+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:26.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:26.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:27.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:27.339+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:27.961 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:27.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:28.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:28.324+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:28.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:28.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:29.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:29.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:29.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:29.867+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:30.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:30.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:30.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:30.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:31.433 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:31.432+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:31.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:31.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:32.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:32.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:32.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:32.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:33.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:33.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:33.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:33.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:34.528 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:34.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:34.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:34.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:35.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:35.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:35.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:35.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:36.464 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:36.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:36.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:36.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:37.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:37.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:37.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:37.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:38.511 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:38.510+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:38.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:38.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:39.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:39.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:39.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:39.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:40.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:40.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:40.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:40.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:41.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:41.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:41.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:41.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:42.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:42.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:42.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:42.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:43.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:43.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:43.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:43.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:44.508 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:44.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:44.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:44.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:45.463 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:45.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:45.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:45.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:46.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:46.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:46.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:46.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:47.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:47.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:47.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:47.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:48.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:48.510+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:48.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:48.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:49.500 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:49.499+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:49.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:49.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:50.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:50.527+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:50.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:50.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:51.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:51.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:51.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:51.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:52.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:52.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:52.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:52.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:53.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:53.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:53.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:53.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:54.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:54.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:54.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:54.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:55.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:55.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:55.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:55.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:56.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:56.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:56.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:56.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:57.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:57.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:57.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:57.792+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:58.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:58.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:58.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:58.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:59.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:00:59.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:00:59.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:00:59.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:00.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:00.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:00.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:00.824+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:01.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:01.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:01.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:01.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:02.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:02.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:02.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:02.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:03.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:03.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:04.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:03.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:04.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:04.617+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:04.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:04.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:05.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:05.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:05.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:05.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:06.662 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:06.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:06.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:06.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:07.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:07.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:07.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:07.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:08.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:08.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:08.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:08.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:09.597 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:09.597+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:09.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:09.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:10.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:10.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:10.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:10.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:11.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:11.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:11.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:11.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:12.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:12.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:12.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:12.856+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:13.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:13.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:13.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:13.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:14.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:14.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:14.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:14.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:15.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:15.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:15.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:15.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:16.558 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:16.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:16.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:16.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:17.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:17.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:17.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:17.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:18.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:18.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:18.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:18.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:19.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:19.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:19.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:19.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:20.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:20.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:20.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:20.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:21.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:21.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:21.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:21.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:22.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:22.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:22.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:22.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:23.459 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:23.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:23.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:23.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:24.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:24.446+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:24.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:24.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:25.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:25.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:25.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:25.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:26.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:26.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:26.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:26.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:27.444 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:27.444+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:27.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:27.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:28.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:28.403+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:28.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:28.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:29.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:29.450+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:30.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:30.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:30.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:30.442+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:31.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:31.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:31.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:31.424+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:32.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:32.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:32.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:32.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:33.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:33.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:33.501 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:33.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:34.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:34.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:34.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:34.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:35.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:35.026+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:35.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:35.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:36.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:36.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:36.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:36.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:37.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:37.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:37.569 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:37.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:38.128 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:38.128+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:38.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:38.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:39.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:39.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:39.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:39.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:40.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:40.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:40.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:40.490+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:41.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:41.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:41.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:41.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:42.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:42.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:42.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:42.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:43.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:43.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:43.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:43.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:44.098 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:44.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:44.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:44.554+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:45.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:45.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:45.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:45.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:46.089 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:46.089+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:46.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:46.539+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:47.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:47.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:47.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:47.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:48.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:48.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:48.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:48.530+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:49.066 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:49.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:49.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:49.547+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:50.109 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:50.109+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:50.498 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:50.498+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:51.152 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:51.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:51.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:51.455+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:52.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:52.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:52.430 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:52.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:53.069 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:53.070+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:53.468 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:53.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:54.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:54.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:54.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:54.435+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:55.127 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:55.128+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:55.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:55.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:56.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:56.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:56.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:56.429+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:57.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:57.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:57.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:57.412+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:58.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:58.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:58.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:58.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:59.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:01:59.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:01:59.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:01:59.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:00.183 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:00.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:00.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:00.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:01.215 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:01.215+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:01.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:01.405+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:02.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:02.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:02.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:02.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:03.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:03.147+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:03.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:03.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:04.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:04.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:04.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:04.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:05.111 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:05.111+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:05.426 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:05.427+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:06.138 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:06.139+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:06.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:06.400+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:07.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:07.170+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:07.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:07.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:08.129 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:08.129+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:08.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:08.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:09.132 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:09.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:09.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:09.340+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:10.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:10.144+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:10.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:10.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:11.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:11.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:11.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:11.378+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:12.106 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:12.106+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:12.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:12.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:13.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:13.087+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:13.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:13.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:14.132 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:14.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:14.441 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:14.441+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:15.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:15.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:15.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:15.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:16.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:16.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:16.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:16.423+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:17.139 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:17.138+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:17.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:17.389+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:18.094 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:18.094+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:18.358 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:18.358+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:19.143 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:19.142+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:19.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:19.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:20.140 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:20.140+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:20.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:20.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:21.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:21.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:21.400 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:21.399+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:22.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:22.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:22.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:22.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:23.149 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:23.148+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:23.420 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:23.420+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:24.121 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:24.120+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:24.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:24.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:25.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:25.171+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:25.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:25.361+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:26.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:26.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:26.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:26.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:27.227 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:27.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:27.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:27.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:28.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:28.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:28.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:28.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:29.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:29.222+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:29.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:29.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:30.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:30.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:30.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:30.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:31.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:31.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:31.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:31.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:32.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:32.279+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:32.296 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:32.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:33.289 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:33.288+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:33.289 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:33.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:34.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:34.266+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:34.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:34.307+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:35.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:35.263+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:35.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:35.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:36.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:36.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:36.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:36.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:37.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:37.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:37.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:37.323+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:38.317 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:38.316+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:38.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:38.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:39.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:39.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:39.350 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:39.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:40.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:40.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:40.382 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:40.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:41.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:41.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:41.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:41.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:42.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:42.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:42.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:42.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:43.364 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:43.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:43.404 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:43.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:44.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:44.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:44.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:44.396+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:45.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:45.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:45.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:45.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:46.359 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:46.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:46.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:46.436+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:47.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:47.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:47.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:47.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:48.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:48.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:48.457 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:48.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:49.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:49.404+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:49.434 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:49.434+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:50.390 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:50.390+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:50.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:50.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:51.355 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:51.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:51.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:51.409+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:52.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:52.366+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:52.403 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:52.402+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:53.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:53.410+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:53.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:53.417+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:54.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:54.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:54.409 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:54.408+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:55.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:55.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:55.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:55.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:56.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:56.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:56.435 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:56.435+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:57.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:57.295+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:57.477 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:57.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:58.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:58.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:58.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:58.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:59.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:02:59.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:02:59.512 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:02:59.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:00.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:00.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:00.525 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:00.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:01.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:01.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:01.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:01.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:02.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:02.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:02.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:02.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:03.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:03.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:03.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:03.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:04.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:04.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:04.598 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:04.597+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:05.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:05.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:05.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:05.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:06.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:06.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:06.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:06.603+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:07.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:07.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:07.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:07.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:08.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:08.299+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:08.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:08.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:09.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:09.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:09.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:09.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:10.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:10.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:10.616 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:10.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:11.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:11.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:11.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:11.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:12.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:12.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:12.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:12.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:13.162 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:13.161+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:13.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:13.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:14.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:14.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:14.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:14.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:15.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:15.123+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:15.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:15.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:16.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:16.166+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:16.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:16.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:17.166 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:17.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:17.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:17.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:18.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:18.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:18.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:18.732+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:19.144 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:19.144+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:19.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:19.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:20.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:20.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:20.687 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:20.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:21.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:21.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:21.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:21.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:22.201 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:22.202+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:22.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:22.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:23.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:23.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:23.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:23.698+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:24.168 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:24.168+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:24.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:24.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:25.157 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:25.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:25.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:25.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:26.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:26.182+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:26.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:26.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:27.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:27.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:27.693 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:27.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:28.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:28.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:28.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:28.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:29.199 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:29.199+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:29.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:29.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:30.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:30.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:30.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:30.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:31.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:31.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:31.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:31.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:32.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:32.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:32.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:32.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:33.214 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:33.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:33.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:33.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:34.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:34.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:34.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:34.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:35.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:35.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:35.727 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:35.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:36.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:36.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:36.769 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:36.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:37.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:37.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:37.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:37.767+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:38.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:38.305+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:38.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:38.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:39.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:39.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:39.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:39.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:40.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:40.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:40.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:40.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:41.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:41.214+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:41.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:41.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:42.178 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:42.178+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:42.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:42.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:43.160 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:43.160+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:43.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:43.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:44.200 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:44.200+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:44.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:44.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:45.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:45.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:45.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:45.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:46.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:46.205+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:46.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:46.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:47.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:47.220+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:47.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:47.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:48.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:48.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:48.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:48.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:49.183 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:49.183+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:49.633 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:49.633+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:50.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:50.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:50.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:50.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:51.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:51.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:51.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:51.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:52.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:52.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:52.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:52.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:53.220 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:53.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:53.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:53.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:54.239 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:54.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:54.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:54.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:55.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:55.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:55.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:55.756+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:56.242 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:56.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:56.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:56.802+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:57.224 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:57.225+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:57.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:57.790+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:58.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:58.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:58.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:58.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:59.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:03:59.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:03:59.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:03:59.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:00.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:00.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:00.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:00.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:01.280 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:01.280+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:01.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:01.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:02.258 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:02.258+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:02.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:02.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:03.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:03.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:03.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:03.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:04.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:04.206+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:04.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:04.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:05.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:05.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:05.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:05.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:06.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:06.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:06.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:06.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:07.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:07.296+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:07.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:07.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:08.321 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:08.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:09.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:09.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:09.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:09.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:10.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:10.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:10.329 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:10.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:11.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:11.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:11.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:11.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:12.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:12.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:12.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:12.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:13.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:13.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:13.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:13.301+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:13.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:13.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:14.292 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:14.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:15.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:15.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:15.315 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:15.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:16.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:16.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:16.312 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:16.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:17.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:17.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:17.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:17.330+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:18.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:18.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:18.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:18.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:19.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:19.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:19.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:19.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:20.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:20.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:20.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:20.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:21.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:21.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:21.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:21.336+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:22.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:22.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:22.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:22.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:23.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:23.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:23.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:23.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:24.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:24.064+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:24.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:24.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:25.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:25.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:25.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:25.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:26.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:26.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:26.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:26.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:27.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:27.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:27.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:27.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:28.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:28.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:28.306 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:28.306+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:29.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:29.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:29.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:29.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:30.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:30.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:30.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:30.289+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:31.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:31.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:31.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:31.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:32.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:32.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:32.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:32.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:33.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:33.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:33.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:33.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:34.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:34.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:34.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:34.326+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:35.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:35.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:35.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:35.343+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:36.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:36.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:36.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:36.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:37.273 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:37.272+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:37.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:37.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:38.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:38.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:38.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:38.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:39.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:39.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:39.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:39.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:40.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:40.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:40.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:40.292+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:41.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:41.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:41.272 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:41.271+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:42.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:42.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:42.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:42.284+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:43.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:43.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:43.244 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:43.244+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:44.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:44.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:44.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:44.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:45.147 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:45.146+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:45.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:45.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:46.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:46.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:46.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:46.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:47.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:47.139+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:47.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:47.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:48.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:48.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:48.288 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:48.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:49.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:49.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:49.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:49.263+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:50.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:50.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:50.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:50.286+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:51.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:51.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:51.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:51.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:52.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:52.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:52.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:52.251+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:53.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:53.282+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:53.294 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:53.293+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:54.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:54.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:54.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:54.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:55.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:55.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:55.305 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:55.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:56.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:56.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:56.349 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:56.348+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:57.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:57.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:57.362 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:57.362+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:58.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:58.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:58.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:58.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:59.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:04:59.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:04:59.374 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:04:59.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:00.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:00.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:00.354 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:00.353+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:01.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:01.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:01.379 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:01.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:02.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:02.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:02.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:02.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:03.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:03.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:03.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:03.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:04.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:04.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:04.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:04.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:05.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:05.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:05.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:05.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:06.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:06.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:06.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:06.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:07.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:07.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:07.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:07.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:08.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:08.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:08.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:08.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:09.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:09.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:09.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:09.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:10.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:10.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:10.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:10.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:11.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:11.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:11.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:11.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:12.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:12.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:12.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:12.444+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:12.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:12.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:13.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:13.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:14.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:14.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:14.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:14.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:15.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:15.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:15.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:15.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:16.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:16.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:16.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:16.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:17.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:17.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:17.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:17.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:18.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:18.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:18.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:18.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:18.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:18.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:19.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:19.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:19.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:19.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:20.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:20.335+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:21.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:21.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:21.293 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:21.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:21.987 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:21.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:22.290 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:22.290+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:23.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:23.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:23.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:23.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:23.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:23.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:24.324 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:24.325+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:25.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:25.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:25.358 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:25.358+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:26.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:26.054+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:26.355 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:26.355+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:27.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:27.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:27.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:27.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:28.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:28.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:28.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:28.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:29.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:29.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:29.395 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:29.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:30.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:30.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:30.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:30.347+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:30.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:30.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:31.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:31.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:31.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:31.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:32.360 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:32.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:32.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:32.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:33.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:33.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:33.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:33.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:34.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:34.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:34.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:34.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:35.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:35.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:35.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:35.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:36.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:36.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:36.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:36.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:37.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:37.394+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:37.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:37.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:38.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:38.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:38.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:38.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:39.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:39.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:39.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:39.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:40.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:40.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:40.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:40.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:41.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:41.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:41.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:41.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:42.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:42.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:42.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:42.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:43.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:43.444+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:43.982 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:43.983+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:44.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:44.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:44.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:44.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:45.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:45.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:45.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:45.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:46.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:46.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:46.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:46.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:47.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:47.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:47.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:47.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:48.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:48.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:48.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:48.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:49.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:49.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:49.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:49.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:50.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:50.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:50.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:50.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:51.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:51.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:51.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:51.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:52.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:52.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:52.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:52.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:53.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:53.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:53.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:53.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:54.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:54.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:54.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:54.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:55.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:55.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:55.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:55.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:56.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:56.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:56.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:56.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:57.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:57.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:57.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:57.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:58.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:58.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:58.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:58.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:59.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:05:59.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:05:59.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:05:59.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:00.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:00.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:00.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:00.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:01.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:01.439+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:01.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:01.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:02.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:02.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:02.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:02.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:03.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:03.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:03.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:03.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:04.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:04.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:04.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:04.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:05.448 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:05.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:05.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:05.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:06.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:06.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:06.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:06.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:07.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:07.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:07.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:07.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:08.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:08.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:08.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:08.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:09.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:09.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:09.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:09.885+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:10.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:10.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:10.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:10.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:11.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:11.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:11.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:11.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:12.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:12.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:12.951 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:12.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:13.514 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:13.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:13.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:13.970+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:14.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:14.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:14.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:14.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:15.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:15.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:16.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:16.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:16.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:16.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:16.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:16.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:17.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:17.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:17.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:17.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:18.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:18.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:18.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:18.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:19.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:19.481+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:19.974 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:19.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:20.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:20.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:20.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:20.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:21.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:21.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:21.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:21.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:22.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:22.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:22.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:22.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:23.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:23.610+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:23.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:23.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:24.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:24.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:24.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:24.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:25.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:25.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:25.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:25.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:26.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:26.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:26.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:26.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:27.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:27.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:27.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:27.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:28.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:28.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:28.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:28.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:29.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:29.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:29.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:29.852+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:30.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:30.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:30.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:30.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:31.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:31.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:31.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:31.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:32.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:32.602+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:32.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:32.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:33.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:33.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:33.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:33.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:34.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:34.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:34.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:34.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:35.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:35.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:35.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:35.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:36.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:36.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:36.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:36.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:37.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:37.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:37.847 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:37.846+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:38.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:38.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:38.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:38.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:39.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:39.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:39.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:39.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:40.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:40.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:40.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:40.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:41.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:41.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:41.932 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:41.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:42.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:42.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:42.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:42.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:43.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:43.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:43.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:43.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:44.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:44.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:44.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:44.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:45.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:45.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:45.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:45.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:46.740 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:46.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:46.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:46.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:47.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:47.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:47.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:47.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:48.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:48.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:48.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:48.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:49.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:49.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:49.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:49.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:50.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:50.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:50.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:50.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:51.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:51.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:51.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:51.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:52.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:52.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:52.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:52.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:53.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:53.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:54.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:54.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:54.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:54.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:55.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:55.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:55.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:55.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:56.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:56.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:56.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:56.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:57.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:57.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:57.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:57.657+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:58.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:58.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:58.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:58.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:59.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:06:59.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:06:59.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:06:59.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:00.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:00.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:00.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:00.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:01.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:01.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:01.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:01.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:02.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:02.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:02.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:02.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:03.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:03.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:03.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:03.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:04.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:04.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:04.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:04.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:05.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:05.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:05.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:05.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:06.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:06.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:06.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:06.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:07.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:07.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:07.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:07.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:08.065 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:08.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:08.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:08.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:09.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:09.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:09.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:09.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:10.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:10.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:10.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:10.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:11.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:11.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:11.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:11.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:12.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:12.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:12.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:12.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:13.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:13.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:13.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:13.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:14.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:14.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:14.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:14.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:15.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:15.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:15.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:15.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:16.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:16.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:16.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:16.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:17.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:17.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:17.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:17.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:18.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:18.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:18.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:18.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:19.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:19.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:19.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:19.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:20.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:20.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:20.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:20.586+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:21.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:21.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:21.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:21.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:22.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:22.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:22.504 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:22.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:23.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:23.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:23.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:23.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:24.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:24.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:24.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:24.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:25.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:25.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:25.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:25.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:26.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:26.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:26.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:26.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:27.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:27.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:27.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:27.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:28.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:28.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:28.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:28.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:29.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:29.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:29.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:29.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:30.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:30.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:30.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:30.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:31.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:31.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:31.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:31.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:32.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:32.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:32.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:32.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:33.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:33.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:33.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:33.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:34.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:34.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:34.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:34.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:35.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:35.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:35.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:35.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:36.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:36.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:36.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:36.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:37.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:37.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:37.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:37.473+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:38.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:38.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:38.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:38.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:39.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:39.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:39.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:39.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:40.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:40.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:40.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:40.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:41.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:41.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:41.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:41.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:42.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:42.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:42.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:42.501+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:43.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:43.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:43.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:43.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:44.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:44.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:44.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:44.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:45.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:45.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:45.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:45.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:46.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:46.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:46.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:46.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:47.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:47.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:47.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:47.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:48.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:48.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:48.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:48.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:49.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:49.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:49.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:49.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:50.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:50.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:50.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:50.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:51.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:51.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:51.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:51.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:52.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:52.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:52.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:52.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:53.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:53.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:53.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:53.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:54.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:54.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:54.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:54.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:54.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:54.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:55.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:55.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:55.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:55.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:56.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:56.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:56.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:56.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:57.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:57.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:57.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:57.931+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:58.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:58.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:58.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:58.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:59.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:07:59.401+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:07:59.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:07:59.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:00.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:00.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:01.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:01.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:01.445 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:01.444+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:02.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:02.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:02.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:02.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:03.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:03.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:03.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:03.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:04.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:04.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:04.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:04.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:05.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:05.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:05.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:05.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:06.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:06.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:06.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:06.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:07.041 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:07.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:07.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:07.449+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:08.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:08.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:08.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:08.421+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:08.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:08.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:09.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:09.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:10.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:10.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:10.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:10.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:11.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:11.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:11.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:11.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:12.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:12.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:12.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:12.404+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:13.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:13.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:13.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:13.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:14.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:14.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:14.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:14.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:15.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:15.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:15.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:15.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:16.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:16.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:16.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:16.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:17.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:17.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:17.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:17.488+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:18.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:18.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:18.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:18.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:19.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:19.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:19.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:19.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:20.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:20.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:20.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:20.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:21.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:21.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:21.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:21.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:22.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:22.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:22.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:22.485+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:23.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:23.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:23.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:23.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:24.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:24.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:24.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:24.476+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:25.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:25.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:25.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:25.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:26.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:26.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:26.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:26.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:27.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:27.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:27.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:27.449+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:28.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:28.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:28.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:28.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:29.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:29.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:29.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:29.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:30.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:30.284+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:30.478 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:30.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:31.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:31.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:31.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:31.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:32.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:32.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:32.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:32.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:33.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:33.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:33.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:33.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:34.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:34.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:34.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:34.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:35.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:35.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:35.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:35.401+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:36.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:36.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:36.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:36.390+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:37.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:37.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:37.392 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:37.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:38.299 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:38.298+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:38.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:38.389+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:39.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:39.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:39.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:39.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:40.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:40.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:40.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:40.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:41.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:41.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:41.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:41.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:42.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:42.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:42.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:42.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:43.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:43.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:43.475 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:43.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:44.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:44.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:44.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:44.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:45.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:45.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:45.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:45.444+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:46.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:46.220+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:46.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:46.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:47.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:47.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:47.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:47.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:48.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:48.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:48.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:48.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:49.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:49.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:49.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:49.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:50.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:50.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:50.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:50.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:51.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:51.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:51.519 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:51.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:52.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:52.197+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:52.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:52.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:53.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:53.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:53.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:53.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:54.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:54.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:54.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:54.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:55.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:55.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:55.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:55.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:56.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:56.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:56.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:56.566+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:57.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:57.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:57.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:57.530+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:58.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:58.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:58.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:58.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:59.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:08:59.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:08:59.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:08:59.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:00.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:00.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:00.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:00.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:01.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:01.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:01.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:01.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:02.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:02.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:02.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:02.415+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:03.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:03.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:03.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:03.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:04.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:04.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:04.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:04.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:05.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:05.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:05.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:05.361+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:06.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:06.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:06.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:06.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:07.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:07.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:07.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:07.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:08.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:08.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:08.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:08.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:09.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:09.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:09.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:09.423+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:10.272 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:10.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:10.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:10.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:11.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:11.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:11.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:11.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:12.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:12.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:12.424 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:12.424+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:13.244 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:13.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:13.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:13.377+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:14.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:14.269+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:14.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:14.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:15.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:15.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:15.366 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:15.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:16.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:16.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:16.346 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:16.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:17.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:17.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:17.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:17.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:18.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:18.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:18.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:18.342+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:19.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:19.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:19.337 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:19.337+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:20.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:20.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:20.298 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:20.298+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:21.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:21.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:21.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:21.343+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:22.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:22.266+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:22.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:22.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:23.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:23.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:23.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:23.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:24.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:24.274+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:24.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:24.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:25.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:25.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:25.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:25.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:26.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:26.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:26.257 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:26.258+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:27.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:27.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:27.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:27.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:28.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:28.246+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:28.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:28.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:29.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:29.260+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:29.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:29.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:30.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:30.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:30.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:30.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:31.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:31.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:31.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:31.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:32.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:32.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:32.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:32.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:33.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:33.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:33.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:33.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:34.192 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:34.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:34.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:34.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:35.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:35.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:35.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:35.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:36.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:36.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:36.262 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:36.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:37.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:37.205+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:37.308 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:37.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:38.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:38.185+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:38.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:38.314+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:39.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:39.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:39.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:39.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:40.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:40.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:40.302 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:40.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:41.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:41.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:41.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:41.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:42.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:42.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:42.323 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:42.323+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:43.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:43.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:43.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:43.346+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:44.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:44.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:44.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:44.309+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:45.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:45.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:45.271 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:45.272+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:46.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:46.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:46.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:46.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:47.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:47.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:47.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:47.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:48.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:48.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:48.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:48.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:49.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:49.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:49.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:49.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:50.243 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:50.243+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:50.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:50.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:51.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:51.268+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:51.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:51.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:52.222 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:52.222+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:52.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:52.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:53.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:53.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:53.373 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:53.373+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:54.170 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:54.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:54.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:54.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:55.181 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:55.181+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:55.319 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:55.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:56.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:56.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:56.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:56.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:57.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:57.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:57.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:57.347+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:58.175 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:58.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:58.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:58.379+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:59.138 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:09:59.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:09:59.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:09:59.337+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:00.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:00.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:00.295 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:00.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:01.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:01.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:01.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:01.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:02.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:02.099+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:02.277 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:02.277+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:03.075 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:03.074+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:03.270 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:03.270+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:04.055 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:04.055+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:04.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:04.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:05.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:05.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:05.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:05.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:05.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:05.982+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:06.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:06.182+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:07.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:07.000+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:07.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:07.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:08.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:08.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:08.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:08.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:09.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:09.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:09.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:09.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:09.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:09.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:10.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:10.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:11.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:11.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:11.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:11.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:12.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:11.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:12.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:12.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:13.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:12.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:13.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:13.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:13.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:13.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:14.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:14.129+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:14.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:14.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:15.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:15.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:15.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:15.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:16.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:16.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:16.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:16.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:17.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:17.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:17.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:17.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:18.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:18.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:18.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:18.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:19.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:19.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:19.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:19.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:20.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:20.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:20.884 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:20.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:21.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:21.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:21.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:21.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:22.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:22.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:22.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:22.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:23.225 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:23.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:23.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:23.912+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:24.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:24.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:24.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:24.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:25.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:25.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:25.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:25.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:26.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:26.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:26.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:26.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:27.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:27.167+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:27.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:27.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:28.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:28.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:28.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:28.954+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:29.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:29.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:29.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:29.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:30.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:30.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:30.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:30.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:31.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:31.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:31.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:31.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:32.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:32.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:32.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:32.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:33.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:33.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:33.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:33.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:34.084 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:34.083+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:34.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:34.910+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:35.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:35.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:35.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:35.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:36.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:36.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:36.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:36.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:37.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:37.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:37.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:37.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:38.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:38.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:38.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:38.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:39.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:39.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:39.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:39.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:40.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:40.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:40.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:40.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:41.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:41.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:41.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:41.839+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:42.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:42.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:42.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:42.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:43.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:43.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:43.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:43.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:44.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:44.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:44.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:44.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:45.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:45.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:45.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:45.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:46.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:46.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:46.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:46.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:47.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:47.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:47.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:47.793+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:48.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:48.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:48.790 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:48.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:49.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:49.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:49.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:49.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:49.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:49.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:50.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:50.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:50.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:50.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:51.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:51.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:51.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:51.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:52.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:52.889+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:53.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:53.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:53.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:53.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:53.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:53.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:54.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:54.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:54.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:54.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:55.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:55.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:55.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:55.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:56.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:56.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:56.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:56.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:57.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:57.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:58.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:57.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:58.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:58.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:58.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:58.938+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:59.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:10:59.912+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:10:59.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:10:59.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:00.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:00.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:00.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:00.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:01.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:01.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:01.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:01.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:02.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:02.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:02.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:02.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:03.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:03.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:03.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:03.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:04.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:04.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:04.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:04.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:05.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:05.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:05.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:05.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:06.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:06.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:06.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:06.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:07.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:07.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:07.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:07.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:08.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:08.937+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:08.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:08.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:09.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:09.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:09.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:09.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:10.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:10.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:10.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:10.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:11.857 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:11.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:11.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:11.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:12.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:12.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:12.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:12.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:13.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:13.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:13.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:13.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:14.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:14.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:14.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:14.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:15.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:15.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:15.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:15.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:16.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:16.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:16.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:16.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:17.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:17.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:17.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:17.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:18.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:18.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:18.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:18.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:19.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:19.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:19.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:19.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:20.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:20.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:20.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:20.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:21.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:21.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:21.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:21.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:22.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:22.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:22.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:22.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:23.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:23.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:23.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:23.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:24.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:24.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:24.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:24.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:25.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:25.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:25.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:25.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:26.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:26.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:26.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:26.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:27.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:27.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:27.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:27.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:28.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:28.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:28.795 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:28.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:29.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:29.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:29.790 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:29.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:30.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:30.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:30.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:30.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:31.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:31.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:31.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:31.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:32.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:32.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:32.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:32.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:33.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:33.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:33.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:33.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:34.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:34.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:34.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:34.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:35.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:35.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:35.766 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:35.767+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:36.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:36.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:36.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:36.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:37.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:37.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:37.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:37.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:38.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:38.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:38.814 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:38.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:39.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:39.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:39.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:39.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:40.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:40.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:40.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:40.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:41.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:41.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:41.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:41.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:42.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:42.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:42.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:42.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:43.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:43.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:43.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:43.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:44.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:44.792+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:44.812 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:44.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:45.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:45.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:45.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:45.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:46.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:46.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:46.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:46.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:47.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:47.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:47.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:47.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:48.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:48.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:48.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:48.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:49.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:49.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:49.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:49.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:50.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:50.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:50.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:50.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:51.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:51.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:51.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:51.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:52.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:52.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:52.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:52.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:53.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:53.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:53.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:53.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:54.725 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:54.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:54.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:54.895+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:55.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:55.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:55.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:55.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:56.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:56.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:56.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:56.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:57.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:57.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:57.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:57.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:58.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:58.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:58.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:58.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:59.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:11:59.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:11:59.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:11:59.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:00.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:00.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:00.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:00.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:01.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:01.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:01.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:01.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:02.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:02.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:02.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:02.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:03.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:03.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:03.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:03.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:04.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:04.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:04.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:04.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:05.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:05.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:05.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:05.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:06.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:06.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:06.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:06.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:07.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:07.685+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:07.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:07.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:08.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:08.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:08.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:08.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:09.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:09.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:09.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:09.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:10.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:10.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:10.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:10.844+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:11.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:11.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:11.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:11.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:12.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:12.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:12.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:12.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:13.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:13.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:13.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:13.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:14.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:14.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:14.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:14.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:15.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:15.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:15.785 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:15.785+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:16.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:16.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:16.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:16.764+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:17.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:17.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:17.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:17.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:18.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:18.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:18.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:18.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:19.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:19.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:19.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:19.779+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:20.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:20.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:20.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:20.745+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:21.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:21.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:21.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:21.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:22.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:22.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:22.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:22.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:23.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:23.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:23.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:23.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:24.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:24.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:24.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:24.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:25.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:25.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:25.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:25.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:26.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:26.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:26.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:26.821+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:27.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:27.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:27.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:27.816+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:28.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:28.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:28.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:28.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:29.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:29.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:29.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:29.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:30.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:30.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:30.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:30.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:31.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:31.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:31.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:31.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:32.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:32.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:32.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:32.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:33.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:33.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:33.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:33.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:34.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:34.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:34.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:34.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:35.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:35.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:35.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:35.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:36.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:36.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:36.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:36.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:37.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:37.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:37.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:37.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:38.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:38.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:38.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:38.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:39.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:39.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:39.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:39.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:40.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:40.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:40.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:40.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:41.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:41.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:41.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:41.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:42.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:42.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:42.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:42.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:43.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:43.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:44.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:44.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:44.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:44.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:45.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:45.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:45.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:45.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:46.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:46.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:46.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:46.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:47.121 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:47.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:47.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:47.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:48.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:48.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:48.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:48.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:49.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:49.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:49.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:49.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:50.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:50.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:50.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:50.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:51.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:51.155+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:51.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:51.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:52.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:52.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:52.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:52.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:53.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:53.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:53.849 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:53.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:54.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:54.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:54.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:54.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:55.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:55.191+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:55.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:55.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:56.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:56.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:56.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:56.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:57.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:57.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:57.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:57.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:58.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:58.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:58.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:58.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:59.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:12:59.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:12:59.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:12:59.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:00.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:00.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:00.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:00.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:01.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:01.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:01.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:01.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:02.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:02.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:02.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:02.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:03.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:03.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:03.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:03.623+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:04.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:04.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:04.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:04.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:05.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:05.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:05.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:05.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:06.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:06.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:06.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:06.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:07.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:07.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:07.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:07.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:08.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:08.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:08.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:08.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:09.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:09.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:09.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:09.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:10.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:10.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:10.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:10.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:11.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:11.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:11.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:11.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:12.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:12.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:12.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:12.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:13.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:13.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:13.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:13.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:14.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:14.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:14.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:14.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:15.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:15.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:15.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:15.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:16.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:16.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:16.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:16.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:17.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:17.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:17.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:17.405+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:18.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:18.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:18.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:18.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:19.092 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:19.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:19.387 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:19.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:20.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:20.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:20.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:20.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:21.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:21.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:21.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:21.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:22.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:22.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:22.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:22.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:23.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:23.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:23.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:23.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:24.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:24.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:24.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:24.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:24.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:24.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:25.361 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:25.360+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:26.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:26.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:26.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:26.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:26.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:26.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:27.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:27.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:27.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:27.945+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:28.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:28.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:28.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:28.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:29.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:29.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:29.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:29.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:30.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:30.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:30.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:30.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:31.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:31.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:31.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:31.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:32.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:32.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:32.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:32.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:33.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:33.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:33.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:33.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:34.447 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:34.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:34.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:34.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:35.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:35.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:35.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:35.844+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:36.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:36.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:36.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:36.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:37.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:37.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:37.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:37.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:38.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:38.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:38.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:38.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:39.522 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:39.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:39.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:39.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:40.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:40.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:40.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:40.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:41.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:41.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:41.910 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:41.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:42.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:42.618+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:42.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:42.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:43.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:43.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:43.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:43.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:44.604 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:44.603+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:44.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:44.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:45.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:45.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:45.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:45.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:46.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:46.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:46.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:46.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:47.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:47.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:47.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:47.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:48.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:48.507+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:48.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:48.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:49.549 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:49.548+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:49.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:49.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:50.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:50.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:50.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:50.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:51.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:51.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:51.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:51.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:52.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:52.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:52.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:52.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:53.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:53.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:53.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:53.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:54.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:54.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:54.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:54.874+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:55.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:55.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:55.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:55.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:56.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:56.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:56.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:56.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:57.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:57.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:57.876 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:57.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:58.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:58.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:58.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:58.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:59.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:13:59.569+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:13:59.960 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:13:59.959+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:00.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:00.595+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:00.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:00.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:01.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:01.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:01.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:01.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:02.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:02.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:02.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:02.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:03.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:03.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:03.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:03.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:04.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:04.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:04.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:04.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:05.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:05.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:05.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:05.922+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:06.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:06.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:06.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:06.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:07.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:07.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:07.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:07.858+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:08.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:08.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:08.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:08.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:09.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:09.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:09.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:09.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:10.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:10.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:10.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:10.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:11.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:11.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:11.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:11.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:12.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:12.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:12.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:12.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:13.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:13.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:13.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:13.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:14.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:14.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:14.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:14.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:15.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:15.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:15.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:15.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:16.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:16.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:16.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:16.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:17.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:17.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:17.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:17.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:18.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:18.775+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:18.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:18.786+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:19.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:19.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:19.816 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:19.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:20.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:20.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:20.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:20.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:21.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:21.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:21.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:21.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:22.805 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:22.804+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:22.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:22.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:23.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:23.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:23.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:23.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:24.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:24.848+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:24.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:24.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:25.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:25.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:25.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:25.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:26.828 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:26.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:26.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:26.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:27.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:27.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:27.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:27.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:28.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:28.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:28.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:28.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:29.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:29.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:29.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:29.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:30.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:30.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:30.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:30.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:31.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:31.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:31.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:31.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:32.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:32.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:32.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:32.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:33.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:33.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:33.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:33.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:34.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:34.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:34.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:34.860+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:35.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:35.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:35.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:35.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:36.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:36.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:36.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:36.805+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:37.755 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:37.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:37.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:37.830+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:38.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:38.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:38.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:38.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:39.839 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:39.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:39.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:39.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:40.837 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:40.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:40.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:40.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:41.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:41.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:41.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:41.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:42.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:42.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:42.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:42.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:43.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:43.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:43.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:43.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:44.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:44.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:44.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:44.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:45.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:45.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:45.944 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:45.944+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:46.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:46.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:46.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:46.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:47.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:47.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:47.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:47.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:48.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:48.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:48.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:48.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:49.895 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:49.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:49.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:49.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:50.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:50.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:50.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:50.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:51.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:51.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:51.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:51.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:52.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:52.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:52.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:52.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:53.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:53.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:53.888 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:53.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:54.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:54.904+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:54.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:54.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:55.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:55.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:55.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:55.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:56.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:56.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:56.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:56.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:57.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:57.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:57.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:57.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:58.841 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:58.841+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:58.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:58.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:59.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:14:59.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:14:59.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:14:59.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:00.782 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:00.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:00.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:00.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:01.809 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:01.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:01.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:01.979+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:02.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:02.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:02.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:02.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:03.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:03.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:03.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:03.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:04.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:04.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:05.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:05.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:05.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:05.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:06.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:06.081+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:06.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:06.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:07.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:07.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:07.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:07.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:08.091 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:08.091+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:08.838 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:08.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:09.105 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:09.105+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:09.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:09.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:10.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:10.155+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:10.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:10.830+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:11.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:11.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:11.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:11.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:12.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:12.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:12.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:12.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:13.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:13.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:13.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:13.815+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:14.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:14.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:14.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:14.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:15.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:15.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:15.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:15.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:16.193 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:16.193+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:16.768 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:16.768+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:17.148 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:17.149+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:17.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:17.795+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:18.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:18.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:18.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:18.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:19.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:19.080+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:19.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:19.813+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:20.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:20.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:20.772 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:20.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:21.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:21.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:21.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:21.812+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:22.044 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:22.044+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:22.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:22.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:23.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:23.037+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:23.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:23.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:24.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:24.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:24.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:24.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:25.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:25.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:25.848 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:25.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:26.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:26.039+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:26.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:26.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:26.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:26.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:27.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:27.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:27.979 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:27.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:28.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:28.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:28.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:28.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:29.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:29.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:29.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:29.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:30.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:30.837+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:31.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:31.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:31.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:31.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:32.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:32.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:32.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:32.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:32.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:32.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:33.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:33.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:33.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:33.973+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:34.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:34.821+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:34.984 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:34.983+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:35.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:35.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:35.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:35.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:36.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:36.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:36.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:36.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:37.878 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:37.878+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:37.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:37.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:38.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:38.919+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:38.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:38.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:39.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:39.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:39.977 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:39.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:40.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:40.894+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:41.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:41.000+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:41.928 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:41.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:41.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:41.965+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:42.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:42.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:42.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:42.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:43.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:43.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:43.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:43.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:44.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:44.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:44.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:44.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:45.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:45.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:45.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:45.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:46.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:46.912+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:46.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:46.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:47.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:47.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:47.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:47.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:48.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:48.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:48.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:48.954+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:49.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:49.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:49.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:49.937+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:50.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:50.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:50.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:50.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:51.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:51.849+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:51.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:51.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:52.879 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:52.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:52.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:52.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:53.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:53.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:53.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:53.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:54.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:54.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:54.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:54.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:55.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:55.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:55.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:55.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:56.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:56.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:56.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:56.914+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:57.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:57.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:57.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:57.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:58.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:58.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:58.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:58.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:59.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:15:59.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:15:59.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:15:59.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:00.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:00.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:00.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:00.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:01.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:01.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:01.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:01.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:02.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:02.803+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:02.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:02.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:03.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:03.792+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:03.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:03.936+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:04.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:04.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:04.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:04.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:05.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:05.840+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:05.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:05.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:06.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:06.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:06.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:06.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:07.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:07.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:07.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:07.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:08.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:08.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:08.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:08.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:09.801 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:09.800+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:09.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:09.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:10.845 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:10.844+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:10.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:10.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:11.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:11.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:11.935 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:11.934+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:12.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:12.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:12.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:12.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:13.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:13.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:13.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:13.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:14.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:14.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:14.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:14.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:15.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:15.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:15.935 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:15.935+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:16.914 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:16.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:16.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:16.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:17.874 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:17.873+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:18.032 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:18.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:18.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:18.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:19.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:19.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:19.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:19.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:20.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:20.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:20.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:20.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:21.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:21.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:21.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:21.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:22.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:22.021+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:22.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:22.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:23.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:23.064+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:23.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:23.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:24.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:24.026+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:24.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:24.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:25.028 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:25.027+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:25.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:25.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:26.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:26.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:26.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:26.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:27.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:27.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:27.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:27.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:27.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:27.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:28.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:28.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:29.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:29.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:29.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:29.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:30.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:30.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:30.905 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:30.904+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:30.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:30.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:31.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:31.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:31.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:31.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:32.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:32.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:32.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:32.973+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:33.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:33.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:34.021 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:34.020+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:34.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:34.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:35.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:35.051+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:35.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:35.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:36.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:36.093+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:36.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:36.920+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:37.121 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:37.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:37.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:37.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:38.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:38.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:38.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:38.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:39.100 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:39.100+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:39.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:39.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:40.132 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:40.133+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:40.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:40.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:41.180 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:41.180+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:41.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:41.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:42.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:42.226+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:42.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:42.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:43.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:43.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:43.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:43.903+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:44.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:44.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:44.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:44.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:45.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:45.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:45.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:45.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:46.281 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:46.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:46.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:46.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:47.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:47.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:47.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:47.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:48.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:48.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:48.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:48.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:49.270 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:49.270+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:50.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:50.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:50.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:50.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:51.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:51.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:51.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:51.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:52.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:52.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:52.253 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:52.253+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:53.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:53.012+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:53.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:53.282+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:54.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:54.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:54.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:54.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:55.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:55.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:55.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:55.263+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:56.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:56.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:56.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:56.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:57.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:57.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:57.320 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:57.320+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:58.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:58.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:58.331 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:58.331+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:59.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:16:59.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:16:59.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:16:59.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:00.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:00.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:00.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:00.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:01.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:01.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:01.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:01.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:02.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:02.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:02.309 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:02.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:03.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:03.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:03.291 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:03.291+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:04.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:04.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:04.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:04.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:05.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:05.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:05.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:05.246+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:06.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:06.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:06.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:06.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:07.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:07.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:07.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:07.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:08.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:08.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:08.183 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:08.183+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:09.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:09.070+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:09.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:09.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:10.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:10.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:10.174 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:10.175+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:11.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:11.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:11.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:11.173+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:12.141 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:12.141+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:12.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:12.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:13.150 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:13.151+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:13.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:13.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:14.108 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:14.108+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:14.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:14.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:15.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:15.060+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:15.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:15.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:16.045 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:16.045+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:16.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:16.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:17.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:17.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:17.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:17.133+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:18.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:18.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:18.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:18.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:18.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:18.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:19.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:19.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:20.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:20.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:20.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:20.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:21.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:21.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:21.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:21.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:22.063 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:22.064+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:22.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:22.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:23.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:23.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:24.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:23.072+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:24.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:24.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:24.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:24.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:24.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:24.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:25.324 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:25.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:25.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:25.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:25.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:25.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:26.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:26.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:27.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:26.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:27.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:27.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:28.256 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:27.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:28.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:28.996+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:29.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:29.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:30.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:30.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:30.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:30.066+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:30.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:30.980+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:31.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:31.044+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:31.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:31.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:32.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:32.061+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:32.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:32.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:33.104 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:33.104+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:33.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:33.879+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:34.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:34.130+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:34.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:34.887+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:35.156 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:35.156+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:35.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:35.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:36.207 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:36.206+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:36.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:36.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:37.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:37.159+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:37.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:37.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:38.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:38.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:38.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:38.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:39.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:39.157+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:39.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:39.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:40.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:40.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:40.844 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:40.843+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:41.147 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:41.147+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:41.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:41.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:42.187 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:42.187+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:42.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:42.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:43.230 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:43.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:43.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:43.867+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:44.203 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:44.202+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:44.906 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:44.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:45.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:45.154+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:45.946 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:45.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:46.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:46.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:46.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:46.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:47.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:47.232+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:47.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:47.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:48.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:48.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:48.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:48.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:49.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:49.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:49.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:49.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:50.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:50.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:50.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:50.924+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:51.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:51.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:51.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:51.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:52.338 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:52.338+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:52.901 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:52.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:53.325 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:53.324+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:53.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:53.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:54.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:54.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:54.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:54.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:55.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:55.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:55.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:55.890+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:56.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:56.248+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:56.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:56.917+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:57.295 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:57.294+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:57.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:57.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:58.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:58.313+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:58.867 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:58.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:59.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:17:59.311+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:17:59.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:17:59.836+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:00.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:00.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:00.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:00.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:01.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:01.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:01.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:01.848+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:02.381 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:02.380+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:02.823 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:02.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:03.389 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:03.388+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:03.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:03.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:04.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:04.409+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:04.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:04.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:05.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:05.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:05.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:05.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:06.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:06.434+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:06.865 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:06.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:07.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:07.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:07.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:07.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:08.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:08.435+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:08.883 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:08.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:09.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:09.414+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:09.919 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:09.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:10.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:10.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:10.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:10.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:11.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:11.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:11.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:11.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:12.369 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:12.369+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:13.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:13.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:13.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:13.415+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:13.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:13.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:14.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:14.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:15.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:15.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:15.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:15.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:16.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:16.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:16.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:16.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:16.991 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:16.990+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:17.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:17.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:17.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:17.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:18.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:18.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:18.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:18.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:19.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:19.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:19.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:19.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:20.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:20.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:21.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:21.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:21.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:21.403+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:22.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:22.009+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:22.436 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:22.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:22.969 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:22.968+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:23.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:23.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:23.980 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:23.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:24.423 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:24.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:24.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:24.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:25.438 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:25.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:25.965 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:25.965+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:26.485 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:26.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:27.009 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:27.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:27.441 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:27.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:28.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:28.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:28.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:28.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:28.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:28.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:29.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:29.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:29.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:29.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:30.428 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:30.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:31.014 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:31.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:31.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:31.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:31.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:31.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:32.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:32.377+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:33.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:33.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:33.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:33.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:34.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:34.011+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:34.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:34.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:35.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:35.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:35.418 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:35.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:36.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:36.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:36.383 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:36.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:36.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:36.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:37.391 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:37.391+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:37.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:37.958+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:38.345 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:38.345+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:38.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:38.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:39.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:39.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:39.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:39.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:40.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:40.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:40.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:40.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:41.443 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:41.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:41.937 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:41.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:42.408 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:42.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:42.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:42.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:43.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:43.372+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:43.945 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:43.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:44.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:44.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:44.940 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:44.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:45.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:45.383+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:45.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:45.908+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:46.356 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:46.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:46.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:46.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:47.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:47.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:47.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:47.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:48.326 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:48.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:48.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:48.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:49.286 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:49.286+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:49.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:49.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:50.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:50.239+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:50.915 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:50.915+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:51.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:51.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:51.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:51.943+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:52.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:52.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:52.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:52.936+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:53.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:53.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:53.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:53.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:54.246 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:54.247+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:54.939 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:54.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:55.250 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:55.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:55.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:55.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:56.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:56.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:56.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:56.973+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:57.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:57.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:57.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:57.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:58.185 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:58.185+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:58.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:58.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:59.234 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:18:59.234+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:18:59.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:18:59.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:00.204 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:00.204+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:00.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:00.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:01.232 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:01.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:01.984 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:01.984+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:02.194 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:02.194+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:02.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:02.953+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:03.223 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:03.223+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:03.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:03.928+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:04.206 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:04.206+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:04.904 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:04.904+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:05.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:05.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:05.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:05.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:06.238 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:06.238+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:06.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:06.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:07.221 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:07.221+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:07.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:07.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:08.190 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:08.191+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:08.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:08.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:09.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:09.229+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:09.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:09.802+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:10.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:10.237+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:10.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:10.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:11.226 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:11.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:11.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:11.767+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:12.189 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:12.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:12.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:12.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:13.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:13.202+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:13.776 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:13.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:14.192 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:14.192+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:14.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:14.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:15.169 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:15.169+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:15.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:15.782+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:16.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:16.125+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:16.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:16.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:17.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:17.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:17.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:17.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:18.128 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:18.128+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:18.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:18.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:19.164 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:19.164+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:19.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:19.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:20.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:20.132+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:20.713 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:20.713+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:21.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:21.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:21.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:21.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:22.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:22.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:22.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:22.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:23.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:23.087+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:23.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:23.684+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:24.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:24.076+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:24.663 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:24.662+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:25.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:25.072+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:25.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:25.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:26.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:26.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:26.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:26.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:27.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:27.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:27.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:27.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:28.155 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:28.154+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:28.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:28.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:29.198 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:29.198+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:29.618 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:29.618+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:30.153 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:30.152+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:30.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:30.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:31.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:31.114+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:31.604 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:31.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:32.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:32.134+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:32.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:32.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:33.134 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:33.133+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:33.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:33.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:34.167 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:34.167+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:34.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:34.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:35.120 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:35.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:35.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:35.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:36.122 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:36.121+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:36.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:36.562+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:37.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:37.089+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:37.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:37.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:38.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:38.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:38.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:38.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:39.107 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:39.107+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:39.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:39.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:40.065 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:40.064+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:40.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:40.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:41.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:41.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:41.527 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:41.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:42.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:42.058+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:42.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:42.570+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:43.083 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:43.082+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:43.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:43.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:44.034 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:44.033+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:44.570 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:44.569+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:45.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:45.029+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:45.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:45.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:46.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:46.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:46.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:46.530+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:47.038 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:47.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:47.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:47.491+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:48.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:48.020+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:48.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:48.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:48.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:48.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:49.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:49.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:49.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:49.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:50.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:50.539+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:50.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:50.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:51.509 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:51.508+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:51.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:51.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:52.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:52.494+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:52.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:52.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:53.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:53.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:53.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:53.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:54.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:54.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:54.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:54.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:55.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:55.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:55.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:55.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:56.538 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:56.538+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:56.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:56.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:57.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:57.509+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:57.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:57.844+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:58.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:58.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:58.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:58.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:59.559 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:19:59.558+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:19:59.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:19:59.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:00.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:00.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:00.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:00.895+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:01.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:01.524+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:01.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:01.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:02.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:02.545+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:02.858 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:02.857+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:03.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:03.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:03.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:03.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:04.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:04.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:04.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:04.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:05.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:05.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:05.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:05.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:06.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:06.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:06.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:06.957+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:07.590 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:07.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:07.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:07.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:08.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:08.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:08.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:08.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:09.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:09.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:09.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:09.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:10.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:10.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:10.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:10.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:11.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:11.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:11.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:11.936+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:12.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:12.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:12.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:12.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:13.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:13.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:13.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:13.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:14.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:14.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:14.924 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:14.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:15.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:15.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:15.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:15.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:16.519 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:16.518+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:16.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:16.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:17.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:17.497+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:17.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:17.911+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:18.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:18.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:18.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:18.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:19.483 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:19.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:19.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:19.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:20.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:20.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:20.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:20.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:21.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:21.484+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:21.860 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:21.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:22.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:22.502+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:22.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:22.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:23.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:23.484+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:23.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:23.882+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:24.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:24.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:24.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:24.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:25.394 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:25.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:25.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:25.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:26.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:26.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:26.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:26.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:27.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:27.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:27.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:27.911+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:28.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:28.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:28.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:28.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:29.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:29.384+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:29.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:29.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:30.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:30.372+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:30.896 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:30.896+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:31.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:31.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:31.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:31.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:32.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:32.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:32.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:32.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:33.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:33.349+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:33.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:33.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:34.325 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:34.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:34.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:34.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:35.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:35.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:35.912 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:35.912+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:36.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:36.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:36.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:36.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:37.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:37.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:37.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:37.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:38.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:38.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:38.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:38.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:39.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:39.380+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:39.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:39.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:40.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:40.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:40.888 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:40.888+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:41.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:41.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:41.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:41.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:42.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:42.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:42.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:42.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:43.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:43.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:43.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:43.894+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:44.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:44.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:44.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:44.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:45.351 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:45.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:45.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:45.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:46.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:46.389+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:46.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:46.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:47.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:47.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:47.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:47.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:48.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:48.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:48.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:48.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:49.354 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:49.355+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:49.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:49.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:50.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:50.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:50.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:50.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:51.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:51.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:51.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:51.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:52.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:52.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:52.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:52.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:53.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:53.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:53.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:53.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:54.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:54.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:54.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:54.884+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:55.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:55.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:55.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:55.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:56.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:56.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:56.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:56.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:57.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:57.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:57.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:57.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:58.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:58.339+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:58.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:58.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:59.351 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:20:59.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:20:59.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:20:59.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:00.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:00.327+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:00.967 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:00.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:01.328 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:01.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:01.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:01.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:02.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:02.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:03.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:03.038+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:03.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:03.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:04.014 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:04.014+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:04.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:04.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:04.993 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:04.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:05.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:05.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:05.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:05.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:06.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:06.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:06.938 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:06.938+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:07.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:07.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:07.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:07.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:08.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:08.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:08.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:08.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:09.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:09.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:09.980 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:09.980+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:10.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:10.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:10.967 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:10.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:11.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:11.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:12.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:12.011+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:12.264 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:12.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:12.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:12.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:13.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:13.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:13.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:13.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:14.332 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:14.332+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:14.948 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:14.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:15.360 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:15.360+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:15.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:15.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:16.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:16.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:16.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:16.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:17.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:17.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:17.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:17.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:18.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:18.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:18.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:18.933+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:19.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:19.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:19.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:19.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:20.474 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:20.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:20.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:20.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:21.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:21.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:21.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:21.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:22.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:22.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:22.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:22.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:23.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:23.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:23.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:23.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:24.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:24.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:24.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:24.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:25.470 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:25.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:25.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:25.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:26.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:26.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:26.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:26.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:27.419 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:27.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:27.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:27.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:28.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:28.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:28.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:28.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:29.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:29.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:29.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:29.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:30.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:30.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:30.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:30.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:31.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:31.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:31.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:31.947+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:32.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:32.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:32.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:32.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:33.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:33.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:33.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:33.952+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:34.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:34.571+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:34.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:34.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:35.594 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:35.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:36.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:36.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:36.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:36.596+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:37.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:37.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:37.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:37.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:38.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:38.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:38.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:38.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:38.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:38.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:39.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:39.559+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:39.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:39.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:40.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:40.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:40.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:40.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:41.514 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:41.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:41.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:41.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:42.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:42.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:42.917 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:42.917+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:43.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:43.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:43.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:43.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:44.461 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:44.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:44.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:44.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:45.484 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:45.483+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:45.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:45.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:46.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:46.470+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:46.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:46.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:47.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:47.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:47.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:47.912+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:48.446 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:48.445+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:48.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:48.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:49.497 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:49.496+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:49.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:49.897+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:50.494 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:50.493+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:50.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:50.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:51.503 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:51.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:51.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:51.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:52.520 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:52.519+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:52.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:52.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:53.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:53.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:53.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:53.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:54.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:54.573+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:54.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:54.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:55.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:55.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:55.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:55.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:56.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:56.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:56.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:56.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:57.596 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:57.595+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:57.794 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:57.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:58.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:58.583+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:58.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:58.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:59.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:21:59.594+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:21:59.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:21:59.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:00.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:00.590+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:00.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:00.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:01.545 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:01.545+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:01.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:01.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:02.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:02.575+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:02.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:02.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:03.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:03.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:03.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:03.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:04.637 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:04.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:04.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:04.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:05.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:05.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:05.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:05.817+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:06.655 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:06.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:06.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:06.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:07.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:07.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:07.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:07.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:08.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:08.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:08.715 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:08.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:09.704 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:09.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:09.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:09.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:10.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:10.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:10.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:10.736+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:11.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:11.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:11.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:11.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:12.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:12.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:12.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:12.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:13.638 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:13.637+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:13.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:13.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:14.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:14.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:14.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:14.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:15.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:15.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:15.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:15.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:16.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:16.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:16.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:16.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:17.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:17.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:17.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:17.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:18.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:18.659+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:18.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:18.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:19.661 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:19.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:19.737 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:19.736+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:20.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:20.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:20.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:20.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:21.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:21.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:21.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:21.718+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:22.617 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:22.616+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:22.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:22.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:23.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:23.627+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:23.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:23.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:24.610 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:24.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:24.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:24.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:25.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:25.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:25.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:25.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:26.555 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:26.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:26.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:26.646+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:27.577 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:27.577+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:27.613 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:27.613+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:28.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:28.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:28.614 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:28.614+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:29.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:29.554+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:29.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:29.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:30.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:30.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:30.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:30.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:31.596 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:31.596+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:31.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:31.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:32.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:32.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:32.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:32.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:33.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:33.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:33.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:33.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:34.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:34.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:34.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:34.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:35.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:35.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:35.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:35.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:36.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:36.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:36.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:36.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:37.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:37.589+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:37.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:37.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:38.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:38.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:38.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:38.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:39.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:39.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:39.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:39.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:40.595 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:40.595+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:40.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:40.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:41.548 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:41.548+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:41.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:41.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:42.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:42.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:42.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:42.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:43.564 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:43.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:43.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:43.803+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:44.539 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:44.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:44.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:44.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:45.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:45.526+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:45.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:45.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:46.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:46.542+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:46.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:46.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:47.566 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:47.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:47.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:47.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:48.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:48.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:48.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:48.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:49.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:49.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:49.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:49.808+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:50.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:50.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:50.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:50.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:51.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:51.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:51.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:51.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:52.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:52.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:52.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:52.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:53.581 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:53.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:53.835 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:53.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:54.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:54.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:54.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:54.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:55.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:55.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:55.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:55.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:56.599 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:56.599+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:56.867 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:56.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:57.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:57.606+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:57.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:57.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:58.639 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:58.640+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:58.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:58.866+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:59.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:22:59.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:22:59.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:22:59.873+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:00.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:00.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:00.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:00.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:01.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:01.651+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:01.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:01.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:02.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:02.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:02.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:02.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:03.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:03.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:03.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:03.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:04.580 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:04.580+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:04.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:04.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:05.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:05.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:05.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:05.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:06.612 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:06.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:06.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:06.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:07.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:07.660+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:07.935 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:07.935+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:08.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:08.643+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:08.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:08.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:09.673 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:09.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:09.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:09.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:10.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:10.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:10.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:10.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:11.699 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:11.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:11.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:11.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:12.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:12.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:12.851 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:12.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:13.729 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:13.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:13.808 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:13.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:14.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:14.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:14.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:14.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:15.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:15.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:15.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:15.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:16.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:16.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:16.822 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:16.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:17.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:17.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:17.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:17.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:18.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:18.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:18.861 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:18.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:19.820 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:19.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:19.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:19.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:20.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:20.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:20.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:20.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:21.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:21.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:21.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:21.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:22.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:22.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:22.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:22.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:23.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:23.835+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:23.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:23.851+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:24.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:24.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:24.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:24.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:25.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:25.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:25.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:25.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:26.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:26.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:26.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:26.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:27.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:27.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:27.855 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:27.854+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:28.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:28.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:28.821 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:28.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:29.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:29.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:29.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:29.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:30.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:30.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:30.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:30.912+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:31.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:31.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:31.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:31.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:32.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:32.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:32.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:32.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:33.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:33.859+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:33.943 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:33.942+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:34.883 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:34.883+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:34.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:34.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:35.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:35.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:35.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:35.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:36.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:36.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:36.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:36.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:37.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:37.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:37.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:37.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:38.863 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:38.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:38.911 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:38.910+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:39.849 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:39.848+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:39.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:39.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:40.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:40.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:40.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:40.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:41.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:41.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:41.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:41.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:42.820 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:42.820+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:42.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:42.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:43.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:43.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:43.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:43.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:44.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:44.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:44.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:44.838+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:45.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:45.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:45.884 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:45.883+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:46.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:46.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:46.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:46.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:47.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:47.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:47.894 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:47.893+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:48.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:48.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:48.861 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:48.860+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:49.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:49.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:49.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:49.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:50.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:50.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:50.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:50.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:51.773 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:51.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:51.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:51.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:52.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:52.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:52.825 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:52.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:53.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:53.740+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:53.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:53.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:54.694 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:54.693+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:54.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:54.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:55.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:55.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:55.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:55.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:56.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:56.664+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:56.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:56.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:57.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:57.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:57.802 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:57.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:58.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:58.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:58.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:58.766+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:59.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:23:59.669+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:23:59.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:23:59.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:00.652 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:00.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:00.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:00.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:01.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:01.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:01.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:01.764+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:02.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:02.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:02.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:02.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:03.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:03.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:03.721 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:03.720+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:04.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:04.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:04.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:04.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:05.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:05.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:05.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:05.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:06.675 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:06.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:06.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:06.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:07.667 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:07.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:07.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:07.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:08.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:08.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:08.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:08.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:09.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:09.655+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:09.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:09.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:10.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:10.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:10.698 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:10.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:11.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:11.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:11.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:11.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:12.647 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:12.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:12.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:12.722+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:13.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:13.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:13.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:13.721+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:14.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:14.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:14.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:14.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:15.595 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:15.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:15.676 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:15.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:16.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:16.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:16.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:16.652+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:17.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:17.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:17.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:17.688+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:18.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:18.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:18.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:18.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:19.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:19.566+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:19.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:19.748+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:20.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:20.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:20.799 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:20.799+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:21.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:21.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:21.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:21.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:22.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:22.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:22.802 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:22.802+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:23.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:23.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:23.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:23.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:24.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:24.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:24.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:24.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:25.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:25.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:25.807 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:25.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:26.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:26.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:26.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:26.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:27.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:27.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:27.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:27.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:28.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:28.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:28.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:28.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:29.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:29.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:29.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:29.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:30.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:30.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:30.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:30.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:31.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:31.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:31.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:31.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:32.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:32.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:32.634 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:32.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:33.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:33.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:33.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:33.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:34.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:34.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:34.653 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:34.653+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:35.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:35.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:35.681 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:35.681+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:36.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:36.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:36.665 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:36.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:37.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:37.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:37.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:37.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:38.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:38.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:38.677 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:38.677+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:39.632 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:39.632+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:39.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:39.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:40.584 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:40.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:40.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:40.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:41.560 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:41.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:41.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:41.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:42.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:42.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:42.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:42.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:43.542 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:43.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:43.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:43.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:44.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:44.544+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:44.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:44.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:45.574 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:45.575+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:45.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:45.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:46.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:46.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:46.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:46.628+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:47.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:47.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:47.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:47.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:48.534 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:48.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:48.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:48.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:49.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:49.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:49.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:49.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:50.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:50.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:50.608 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:50.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:51.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:51.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:51.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:51.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:52.469 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:52.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:52.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:52.607+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:53.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:53.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:53.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:53.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:54.451 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:54.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:54.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:54.609+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:55.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:55.443+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:55.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:55.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:56.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:56.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:56.664 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:56.665+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:57.425 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:57.425+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:57.691 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:57.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:58.411 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:58.411+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:58.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:58.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:59.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:24:59.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:24:59.685 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:24:59.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:00.417 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:00.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:00.666 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:00.666+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:01.385 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:01.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:01.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:01.710+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:02.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:02.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:02.660 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:02.661+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:03.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:03.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:03.679 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:03.679+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:04.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:04.417+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:04.682 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:04.682+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:05.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:05.373+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:05.647 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:05.647+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:06.405 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:06.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:06.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:06.649+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:07.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:07.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:07.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:07.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:08.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:08.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:08.656 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:08.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:09.426 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:09.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:09.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:09.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:10.390 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:10.390+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:10.683 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:10.683+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:11.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:11.441+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:11.678 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:11.678+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:12.399 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:12.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:12.641 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:12.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:13.364 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:13.364+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:13.630 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:13.630+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:14.402 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:14.401+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:14.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:14.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:15.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:15.401+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:15.583 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:15.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:16.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:16.352+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:16.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:16.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:17.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:17.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:17.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:17.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:18.370 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:18.370+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:18.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:18.615+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:19.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:19.415+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:19.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:19.568+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:20.403 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:20.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:20.588 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:20.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:21.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:21.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:21.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:21.601+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:22.400 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:22.399+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:22.622 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:22.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:23.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:23.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:23.651 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:23.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:24.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:24.367+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:24.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:24.699+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:25.380 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:25.379+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:25.738 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:25.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:26.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:26.422+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:26.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:26.784+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:27.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:27.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:27.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:27.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:28.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:28.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:28.784 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:28.783+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:29.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:29.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:29.779 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:29.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:30.599 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:30.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:30.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:30.774+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:31.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:31.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:31.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:31.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:32.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:32.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:32.705 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:32.705+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:33.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:33.549+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:33.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:33.755+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:34.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:34.571+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:34.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:34.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:35.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:35.536+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:35.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:35.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:36.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:36.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:36.765 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:36.764+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:37.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:37.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:37.746 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:37.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:38.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:38.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:38.794 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:38.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:39.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:39.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:39.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:39.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:40.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:40.469+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:40.804 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:40.803+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:41.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:41.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:41.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:41.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:42.492 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:42.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:42.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:42.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:43.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:43.509+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:43.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:43.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:44.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:44.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:44.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:44.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:45.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:45.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:45.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:45.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:46.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:46.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:46.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:46.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:47.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:47.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:47.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:47.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:48.388 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:48.387+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:48.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:48.975+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:49.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:49.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:50.008 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:50.008+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:50.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:50.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:51.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:51.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:51.372 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:51.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:51.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:51.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:52.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:52.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:52.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:52.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:53.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:53.437+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:53.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:53.929+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:54.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:54.462+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:54.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:54.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:55.433 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:55.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:55.927 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:55.927+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:56.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:56.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:56.880 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:56.880+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:57.412 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:57.411+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:57.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:57.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:58.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:58.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:58.831 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:58.831+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:59.373 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:25:59.372+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:25:59.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:25:59.807+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:00.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:00.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:00.764 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:00.764+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:01.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:01.431+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:01.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:01.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:02.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:02.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:02.522 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-29T21:26:02.523 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T21:26:02.522+0000 7ffb74ff9700 -1 Fail to read '/proc/92656/cmdline' error = (3) No such process 2023-09-29T21:26:02.523 INFO:tasks.workunit.client.0.smithi143.stderr:2023-09-29T21:26:02.522+0000 7ffb74ff9700 -1 received signal: Terminated from (PID: 92656) UID: 1000 2023-09-29T21:26:02.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:02.728+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:03.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:03.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:03.777 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:03.776+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:04.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:04.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:04.818 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:04.818+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:05.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:05.464+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:05.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:05.777+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:06.466 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:06.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:06.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:06.806+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:07.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:07.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:07.843 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:07.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:08.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:08.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:08.825 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:08.822+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:09.541 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:09.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:09.840 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:09.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:10.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:10.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:10.857 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:10.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:11.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:11.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:11.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:11.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:12.570 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:12.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:12.806 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:12.805+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:13.561 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:13.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:13.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:13.810+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:14.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:14.539+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:14.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:14.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:15.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:15.562+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:15.760 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:15.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:16.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:16.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:16.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:16.751+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:17.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:17.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:17.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:17.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:18.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:18.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:18.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:18.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:19.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:19.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:19.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:19.781+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:20.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:20.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:20.743 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:20.743+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:21.656 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:21.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:21.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:21.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:22.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:22.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:22.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:22.740+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:23.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:23.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:23.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:23.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:24.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:24.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:24.789 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:24.789+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:25.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:25.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:25.827 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:25.827+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:26.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:26.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:26.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:26.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:27.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:27.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:27.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:27.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:28.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:28.621+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:28.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:28.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:29.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:29.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:29.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:29.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:30.683 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:30.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:30.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:30.755+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:31.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:31.671+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:31.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:31.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:32.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:32.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:32.745 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:32.746+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:33.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:33.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:33.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:33.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:34.696 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:34.696+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:34.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:34.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:35.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:35.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:35.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:35.739+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:36.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:36.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:36.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:36.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:37.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:37.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:37.775 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:37.775+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:38.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:38.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:38.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:38.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:39.700 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:39.700+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:39.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:39.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:40.709 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:40.709+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:40.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:40.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:41.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:41.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:41.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:41.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:42.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:42.725+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:42.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:42.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:43.741 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:43.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:43.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:43.795+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:44.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:44.753+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:44.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:44.783+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:45.787 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:45.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:45.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:45.793+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:46.752 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:46.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:46.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:46.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:47.788 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:47.788+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:47.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:47.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:48.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:48.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:48.815 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:48.816+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:49.770 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:49.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:49.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:49.819+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:50.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:50.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:50.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:50.865+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:51.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:51.824+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:51.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:51.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:52.811 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:52.811+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:52.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:52.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:53.797 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:53.798+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:53.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:53.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:54.791 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:54.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:54.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:54.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:55.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:55.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:55.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:55.922+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:56.798 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:56.798+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:56.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:56.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:57.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:57.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:57.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:57.983+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:58.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:58.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:58.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:58.991+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:59.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:26:59.801+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:26:59.988 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:26:59.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:00.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:00.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:00.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:00.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:01.852 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:01.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:01.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:01.955+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:02.525 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-09-29T21:27:02.813 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:02.814+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:02.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:02.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:03.830 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:03.830+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:03.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:03.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:04.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:04.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:04.941 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:04.941+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:05.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:05.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:06.008 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:05.947+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:06.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:06.829+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:06.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:06.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:07.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:07.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:07.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:07.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:08.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:08.899+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:08.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:08.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:09.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:09.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:09.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:09.954+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:10.868 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:10.868+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:11.004 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:11.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:11.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:11.889+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:12.046 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:12.046+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:12.924 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:12.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:13.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:13.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:13.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:13.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:14.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:14.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:14.948 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:14.947+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:15.031 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:15.030+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:15.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:15.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:15.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:15.987+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:16.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:16.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:16.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:16.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:17.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:17.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:17.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:17.995+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:18.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:18.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:19.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:19.005+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:19.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:19.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:19.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:19.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:20.925 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:20.925+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:20.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:20.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:21.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:21.948+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:21.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:21.949+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:22.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:22.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:22.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:22.960+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:23.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:23.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:23.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:23.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:24.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:24.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:24.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:24.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:25.900 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:25.900+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:26.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:26.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:26.899 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:26.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:27.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:27.015+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:27.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:27.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:28.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:28.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:28.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:28.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:28.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:28.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:29.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:29.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:30.025 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:30.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:30.970 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:30.969+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:30.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:30.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:31.967 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:31.966+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:32.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:32.023+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:32.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:32.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:33.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:33.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:33.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:33.989+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:33.993 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:33.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:34.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:34.996+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:35.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:35.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:35.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:35.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:36.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:36.039+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:37.003 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:37.002+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:37.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:37.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:37.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:37.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:38.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:38.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:38.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:38.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:39.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:39.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:39.995 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:39.994+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:40.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:40.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:41.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:41.034+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:41.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:41.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:42.000 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:41.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:42.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:42.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:42.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:42.969+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:43.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:43.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:43.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:43.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:44.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:44.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:44.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:44.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:45.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:45.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:45.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:45.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:46.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:46.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:46.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:46.965+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:47.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:47.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:47.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:47.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:48.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:48.120+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:48.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:48.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:49.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:49.088+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:49.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:49.961+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:50.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:50.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:50.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:50.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:51.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:51.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:51.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:51.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:52.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:52.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:52.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:52.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:53.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:53.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:53.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:53.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:54.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:54.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:54.902 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:54.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:55.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:55.196+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:55.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:55.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:56.204 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:56.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:56.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:56.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:57.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:57.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:57.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:57.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:58.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:58.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:58.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:58.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:59.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:27:59.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:27:59.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:27:59.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:00.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:00.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:00.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:00.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:01.259 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:01.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:01.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:01.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:02.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:02.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:02.527 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-09-29T21:28:02.527 INFO:tasks.workunit:Stopping ['rbd/cli_generic.sh'] on client.0... 2023-09-29T21:28:02.527 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-09-29T21:28:02.819 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi143 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c186684db7f9ea0cdb098fc86b350a0fdd26eba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-09-29T21:28:04.047 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d6ea18ea60704a38bff424802b4f69d9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi143 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c186684db7f9ea0cdb098fc86b350a0fdd26eba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-09-29T21:28:04.049 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-29T21:28:04.058 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-09-29T21:28:04.058 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T21:28:04.071 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:02.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:04.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:03.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:04.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:03.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:04.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:04.261+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:28:04.392 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":162,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T19:28:18.074438+0000","last_up_change":"2023-09-29T18:24:57.285879+0000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T18:24:57.558720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T18:25:02.831861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-09-29T18:25:10.287623+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":86744},{"type":"v1","addr":"172.21.15.143:6809","nonce":86744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":86744},{"type":"v1","addr":"172.21.15.143:6811","nonce":86744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":86744},{"type":"v1","addr":"172.21.15.143:6815","nonce":86744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":86744},{"type":"v1","addr":"172.21.15.143:6813","nonce":86744}]},"public_addr":"172.21.15.143:6809/86744","cluster_addr":"172.21.15.143:6811/86744","heartbeat_back_addr":"172.21.15.143:6815/86744","heartbeat_front_addr":"172.21.15.143:6813/86744","state":["exists","up"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":86743},{"type":"v1","addr":"172.21.15.143:6801","nonce":86743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":86743},{"type":"v1","addr":"172.21.15.143:6803","nonce":86743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":86743},{"type":"v1","addr":"172.21.15.143:6807","nonce":86743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":86743},{"type":"v1","addr":"172.21.15.143:6805","nonce":86743}]},"public_addr":"172.21.15.143:6801/86743","cluster_addr":"172.21.15.143:6803/86743","heartbeat_back_addr":"172.21.15.143:6807/86743","heartbeat_front_addr":"172.21.15.143:6805/86743","state":["exists","up"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":86759},{"type":"v1","addr":"172.21.15.143:6817","nonce":86759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":86759},{"type":"v1","addr":"172.21.15.143:6819","nonce":86759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":86759},{"type":"v1","addr":"172.21.15.143:6823","nonce":86759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":86759},{"type":"v1","addr":"172.21.15.143:6821","nonce":86759}]},"public_addr":"172.21.15.143:6817/86759","cluster_addr":"172.21.15.143:6819/86759","heartbeat_back_addr":"172.21.15.143:6823/86759","heartbeat_front_addr":"172.21.15.143:6821/86759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.827743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.725152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.753719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":6,"snaps":[{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T21:28:04.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-29T21:28:04.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:28:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:28:04.766 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:28:04.777 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5650,"stamp":"2023-09-29T21:28:03.196764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34871,"ondisk_log_size":34871,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":784268,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":191424,"kb_avail":282331252,"statfs":{"total":289910292480,"available":289107202048,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":196018176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"159'2384","reported_seq":3269,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074719+0000","last_change":"2023-09-29T18:30:06.801366+0000","last_active":"2023-09-29T19:27:48.074719+0000","last_peered":"2023-09-29T19:27:48.074719+0000","last_clean":"2023-09-29T19:27:48.074719+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T19:27:48.074719+0000","last_undegraded":"2023-09-29T19:27:48.074719+0000","last_fullsized":"2023-09-29T19:27:48.074719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2384,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"159'1917","reported_seq":2794,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074758+0000","last_change":"2023-09-29T18:30:06.801380+0000","last_active":"2023-09-29T19:27:48.074758+0000","last_peered":"2023-09-29T19:27:48.074758+0000","last_clean":"2023-09-29T19:27:48.074758+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T19:27:48.074758+0000","last_undegraded":"2023-09-29T19:27:48.074758+0000","last_fullsized":"2023-09-29T19:27:48.074758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1917,"ondisk_log_size":1917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16450,"ondisk_log_size":16450,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740572,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246288,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":94125552,"statfs":{"total":96636764160,"available":96384565248,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.22,"15min":0.217},"min":{"1min":0.16900000000000001,"5min":0.156,"15min":0.156},"max":{"1min":0.26000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.223,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16},"max":{"1min":0.29399999999999998,"5min":0.45900000000000002,"15min":0.47599999999999998},"last":0.193}]},{"osd":1,"last update":"Fri Sep 29 21:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17199999999999999},"max":{"1min":0.314,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.26400000000000001,"15min":0.26100000000000001},"min":{"1min":0.20999999999999999,"5min":0.19700000000000001,"15min":0.17599999999999999},"max":{"1min":0.28399999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.23300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740572,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.14199999999999999,"5min":0.14000000000000001,"15min":0.123},"max":{"1min":0.36399999999999999,"5min":0.42099999999999999,"15min":0.435},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.21099999999999999,"15min":0.184},"max":{"1min":0.42999999999999999,"5min":0.43099999999999999,"15min":0.502},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.16300000000000001},"max":{"1min":0.378,"5min":0.378,"15min":0.43099999999999999},"last":0.249},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.254,"15min":0.254},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.17799999999999999},"max":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.49199999999999999},"last":0.30499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287796,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94084044,"statfs":{"total":96636764160,"available":96342061056,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.214,"15min":0.20799999999999999},"min":{"1min":0.16800000000000001,"5min":0.14399999999999999,"15min":0.13900000000000001},"max":{"1min":0.24099999999999999,"5min":0.40999999999999998,"15min":0.41299999999999998},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.248,"15min":0.23899999999999999},"min":{"1min":0.20699999999999999,"5min":0.16,"15min":0.16},"max":{"1min":0.27100000000000002,"5min":0.443,"15min":0.44500000000000001},"last":0.246}]},{"osd":2,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16300000000000001},"max":{"1min":0.29699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.28399999999999997,"15min":0.27800000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.20499999999999999},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.31900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:28:04.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:28:04.889 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:04.888+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:05.093 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:28:05.095 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:28:05.106 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5650,"stamp":"2023-09-29T21:28:03.196764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34871,"ondisk_log_size":34871,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":784268,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":191424,"kb_avail":282331252,"statfs":{"total":289910292480,"available":289107202048,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":196018176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"159'2384","reported_seq":3269,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074719+0000","last_change":"2023-09-29T18:30:06.801366+0000","last_active":"2023-09-29T19:27:48.074719+0000","last_peered":"2023-09-29T19:27:48.074719+0000","last_clean":"2023-09-29T19:27:48.074719+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T19:27:48.074719+0000","last_undegraded":"2023-09-29T19:27:48.074719+0000","last_fullsized":"2023-09-29T19:27:48.074719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2384,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"159'1917","reported_seq":2794,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074758+0000","last_change":"2023-09-29T18:30:06.801380+0000","last_active":"2023-09-29T19:27:48.074758+0000","last_peered":"2023-09-29T19:27:48.074758+0000","last_clean":"2023-09-29T19:27:48.074758+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T19:27:48.074758+0000","last_undegraded":"2023-09-29T19:27:48.074758+0000","last_fullsized":"2023-09-29T19:27:48.074758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1917,"ondisk_log_size":1917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16450,"ondisk_log_size":16450,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740572,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246288,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":94125552,"statfs":{"total":96636764160,"available":96384565248,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.22,"15min":0.217},"min":{"1min":0.16900000000000001,"5min":0.156,"15min":0.156},"max":{"1min":0.26000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.223,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16},"max":{"1min":0.29399999999999998,"5min":0.45900000000000002,"15min":0.47599999999999998},"last":0.193}]},{"osd":1,"last update":"Fri Sep 29 21:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17199999999999999},"max":{"1min":0.314,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.26400000000000001,"15min":0.26100000000000001},"min":{"1min":0.20999999999999999,"5min":0.19700000000000001,"15min":0.17599999999999999},"max":{"1min":0.28399999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.23300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740572,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.14199999999999999,"5min":0.14000000000000001,"15min":0.123},"max":{"1min":0.36399999999999999,"5min":0.42099999999999999,"15min":0.435},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.21099999999999999,"15min":0.184},"max":{"1min":0.42999999999999999,"5min":0.43099999999999999,"15min":0.502},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.16300000000000001},"max":{"1min":0.378,"5min":0.378,"15min":0.43099999999999999},"last":0.249},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.254,"15min":0.254},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.17799999999999999},"max":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.49199999999999999},"last":0.30499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287796,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94084044,"statfs":{"total":96636764160,"available":96342061056,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.214,"15min":0.20799999999999999},"min":{"1min":0.16800000000000001,"5min":0.14399999999999999,"15min":0.13900000000000001},"max":{"1min":0.24099999999999999,"5min":0.40999999999999998,"15min":0.41299999999999998},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.248,"15min":0.23899999999999999},"min":{"1min":0.20699999999999999,"5min":0.16,"15min":0.16},"max":{"1min":0.27100000000000002,"5min":0.443,"15min":0.44500000000000001},"last":0.246}]},{"osd":2,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16300000000000001},"max":{"1min":0.29699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.28399999999999997,"15min":0.27800000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.20499999999999999},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.31900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:28:05.107 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-29T21:28:05.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:28:05.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:05.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:05.422 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:28:05.424 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:28:05.435 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5651,"stamp":"2023-09-29T21:28:05.197099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34871,"ondisk_log_size":34871,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":784268,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":191424,"kb_avail":282331252,"statfs":{"total":289910292480,"available":289107202048,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":196018176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001601"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"159'2384","reported_seq":3269,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074719+0000","last_change":"2023-09-29T18:30:06.801366+0000","last_active":"2023-09-29T19:27:48.074719+0000","last_peered":"2023-09-29T19:27:48.074719+0000","last_clean":"2023-09-29T19:27:48.074719+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T19:27:48.074719+0000","last_undegraded":"2023-09-29T19:27:48.074719+0000","last_fullsized":"2023-09-29T19:27:48.074719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2384,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"159'1917","reported_seq":2794,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074758+0000","last_change":"2023-09-29T18:30:06.801380+0000","last_active":"2023-09-29T19:27:48.074758+0000","last_peered":"2023-09-29T19:27:48.074758+0000","last_clean":"2023-09-29T19:27:48.074758+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T19:27:48.074758+0000","last_undegraded":"2023-09-29T19:27:48.074758+0000","last_fullsized":"2023-09-29T19:27:48.074758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1917,"ondisk_log_size":1917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16450,"ondisk_log_size":16450,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740572,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246288,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":94125552,"statfs":{"total":96636764160,"available":96384565248,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.22,"15min":0.217},"min":{"1min":0.16900000000000001,"5min":0.156,"15min":0.156},"max":{"1min":0.26000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.223,"5min":0.23999999999999999,"15min":0.23599999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16},"max":{"1min":0.29399999999999998,"5min":0.45900000000000002,"15min":0.47599999999999998},"last":0.193}]},{"osd":1,"last update":"Fri Sep 29 21:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17199999999999999},"max":{"1min":0.314,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.26400000000000001,"15min":0.26100000000000001},"min":{"1min":0.20999999999999999,"5min":0.19700000000000001,"15min":0.17599999999999999},"max":{"1min":0.28399999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.23300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740572,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.14199999999999999,"5min":0.14000000000000001,"15min":0.123},"max":{"1min":0.36399999999999999,"5min":0.42099999999999999,"15min":0.435},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.21099999999999999,"15min":0.184},"max":{"1min":0.42999999999999999,"5min":0.43099999999999999,"15min":0.502},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23400000000000001,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.16300000000000001},"max":{"1min":0.378,"5min":0.378,"15min":0.43099999999999999},"last":0.249},{"interface":"front","average":{"1min":0.23899999999999999,"5min":0.254,"15min":0.254},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.17799999999999999},"max":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.49199999999999999},"last":0.30499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287796,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94084044,"statfs":{"total":96636764160,"available":96342061056,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.214,"15min":0.20799999999999999},"min":{"1min":0.16800000000000001,"5min":0.14399999999999999,"15min":0.13900000000000001},"max":{"1min":0.24099999999999999,"5min":0.40999999999999998,"15min":0.41299999999999998},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.248,"15min":0.23899999999999999},"min":{"1min":0.20699999999999999,"5min":0.16,"15min":0.16},"max":{"1min":0.27100000000000002,"5min":0.443,"15min":0.44500000000000001},"last":0.246}]},{"osd":2,"last update":"Fri Sep 29 21:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.23300000000000001,"15min":0.22800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16300000000000001},"max":{"1min":0.29699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.28399999999999997,"15min":0.27800000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.20499999999999999},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.31900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:28:05.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T21:28:05.758 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:28:05.759 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":162,"fsid":"210a5970-1bec-4387-ae0f-c041593b74fd","created":"2023-09-29T18:24:47.581820+0000","modified":"2023-09-29T19:28:18.074438+0000","last_up_change":"2023-09-29T18:24:57.285879+0000","last_in_change":"2023-09-29T18:24:48.553467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T18:24:57.558720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T18:25:02.831861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-09-29T18:25:10.287623+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d8bf0b0d-c569-4438-91c9-9604198a9a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":86744},{"type":"v1","addr":"172.21.15.143:6809","nonce":86744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":86744},{"type":"v1","addr":"172.21.15.143:6811","nonce":86744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":86744},{"type":"v1","addr":"172.21.15.143:6815","nonce":86744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":86744},{"type":"v1","addr":"172.21.15.143:6813","nonce":86744}]},"public_addr":"172.21.15.143:6809/86744","cluster_addr":"172.21.15.143:6811/86744","heartbeat_back_addr":"172.21.15.143:6815/86744","heartbeat_front_addr":"172.21.15.143:6813/86744","state":["exists","up"]},{"osd":1,"uuid":"f208ee2f-e70e-4067-862e-5eb63df7af24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":86743},{"type":"v1","addr":"172.21.15.143:6801","nonce":86743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":86743},{"type":"v1","addr":"172.21.15.143:6803","nonce":86743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":86743},{"type":"v1","addr":"172.21.15.143:6807","nonce":86743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":86743},{"type":"v1","addr":"172.21.15.143:6805","nonce":86743}]},"public_addr":"172.21.15.143:6801/86743","cluster_addr":"172.21.15.143:6803/86743","heartbeat_back_addr":"172.21.15.143:6807/86743","heartbeat_front_addr":"172.21.15.143:6805/86743","state":["exists","up"]},{"osd":2,"uuid":"ade283d5-0932-421f-a080-6c4a679eaf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":86759},{"type":"v1","addr":"172.21.15.143:6817","nonce":86759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":86759},{"type":"v1","addr":"172.21.15.143:6819","nonce":86759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":86759},{"type":"v1","addr":"172.21.15.143:6823","nonce":86759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":86759},{"type":"v1","addr":"172.21.15.143:6821","nonce":86759}]},"public_addr":"172.21.15.143:6817/86759","cluster_addr":"172.21.15.143:6819/86759","heartbeat_back_addr":"172.21.15.143:6823/86759","heartbeat_front_addr":"172.21.15.143:6821/86759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.827743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.725152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T18:24:53.753719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":6,"snaps":[{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T21:28:05.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:05.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:06.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:06.200+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:06.773 INFO:tasks.ceph:Scrubbing osd.0 2023-09-29T21:28:06.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-29T21:28:06.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:06.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:07.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:07.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:07.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:07.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:08.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:08.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:08.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:08.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:09.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:09.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:09.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:09.935+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:10.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:10.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:10.933 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:10.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:11.211 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:11.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:11.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:11.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:12.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:12.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:12.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:12.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:13.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:13.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:13.934 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:13.934+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:14.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:14.241+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:14.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:14.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:15.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:15.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:15.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:15.959+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:16.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:16.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:16.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:16.973+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:17.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:17.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:17.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:17.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:18.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:18.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:18.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:18.965+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:19.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:19.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:19.956 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:19.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:20.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:20.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:20.981 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:20.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:21.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:21.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:21.977 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:21.977+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:22.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:22.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:22.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:22.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:23.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:23.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:23.995 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:23.995+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:24.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:24.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:24.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:24.987+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:25.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:25.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:26.002 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:26.003+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:26.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:26.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:27.030 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:27.031+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:27.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:27.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:27.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:27.982+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:28.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:28.066+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:28.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:28.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:29.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:29.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:29.916 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:29.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:30.075 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:30.075+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:30.939 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:30.940+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:31.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:31.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:31.975 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:31.975+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:32.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:32.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:32.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:32.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:32.960 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:32.960+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:33.966 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:33.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:34.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:34.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:34.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:34.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:35.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:35.051+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:35.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:35.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:36.058 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:36.059+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:36.961 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:36.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:37.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:37.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:37.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:37.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:38.117 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:38.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:38.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:38.964+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:39.077 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:39.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:39.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:39.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:40.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:40.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:40.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:40.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:41.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:41.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:41.985 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:41.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:42.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:42.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:42.990 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:42.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:43.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:43.016+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:43.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:43.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:44.017 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:44.017+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:44.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:44.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:45.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:45.052+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:46.029 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:46.029+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:46.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:46.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:47.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:47.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:47.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:47.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:48.007 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:48.007+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:48.052 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:48.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:48.973 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:48.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:49.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:49.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:50.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:50.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:50.023 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:50.024+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:50.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:50.992+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:51.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:51.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:52.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:52.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:52.078 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:52.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:53.012 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:53.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:53.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:53.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:54.055 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:54.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:54.085 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:54.085+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:55.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:55.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:55.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:55.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:55.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:55.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:56.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:56.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:56.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:56.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:57.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:57.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:57.965 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:57.965+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:58.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:58.101+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:58.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:58.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:59.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:28:59.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:28:59.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:28:59.978+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:00.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:00.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:00.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:00.994+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:01.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:01.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:01.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:01.962+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:02.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:02.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:02.924 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:02.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:03.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:03.174+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:03.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:03.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:04.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:04.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:04.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:04.924+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:05.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:05.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:05.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:05.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:06.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:06.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:06.870 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:06.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:07.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:07.097+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:07.887 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:07.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:08.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:08.145+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:08.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:08.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:09.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:09.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:09.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:09.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:10.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:10.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:10.893 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:10.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:11.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:11.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:11.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:11.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:12.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:12.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:12.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:12.915+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:13.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:13.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:13.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:13.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:14.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:14.145+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:14.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:14.892+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:15.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:15.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:15.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:15.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:16.154 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:16.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:16.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:16.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:17.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:17.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:17.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:17.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:18.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:18.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:18.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:18.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:19.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:19.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:19.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:19.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:20.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:20.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:20.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:20.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:21.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:21.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:21.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:21.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:22.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:22.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:22.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:22.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:23.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:23.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:23.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:23.851+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:24.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:24.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:24.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:24.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:25.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:25.035+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:25.881 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:25.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:26.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:26.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:26.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:26.926+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:27.019 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:27.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:27.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:27.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:28.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:28.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:28.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:28.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:28.997 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:28.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:29.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:29.912+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:29.950 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:29.950+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:30.928 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:30.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:30.989 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:30.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:31.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:31.893+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:32.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:32.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:32.864 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:32.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:33.004 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:33.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:33.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:33.837+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:33.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:33.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:34.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:34.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:35.021 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:35.020+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:35.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:35.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:35.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:35.977+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:36.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:36.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:36.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:36.940+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:37.927 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:37.927+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:37.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:37.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:38.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:38.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:38.994 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:38.993+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:39.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:39.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:39.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:39.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:40.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:40.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:40.954 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:40.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:41.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:41.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:41.953 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:41.952+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:42.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:42.900+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:42.938 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:42.937+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:43.856 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:43.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:43.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:43.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:44.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:44.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:44.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:44.962+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:45.910 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:45.909+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:46.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:46.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:46.936 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:46.935+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:47.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:47.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:47.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:47.950+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:48.089 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:48.089+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:48.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:48.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:49.094 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:49.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:49.923 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:49.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:50.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:50.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:50.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:50.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:51.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:51.117+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:51.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:51.951+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:52.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:52.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:52.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:52.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:53.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:53.065+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:53.951 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:53.950+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:54.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:54.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:54.944 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:54.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:55.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:55.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:55.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:55.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:56.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:56.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:57.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:57.008+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:57.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:57.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:58.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:58.059+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:58.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:58.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:59.067 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:29:59.067+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:29:59.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:29:59.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:00.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:00.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:00.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:00.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:01.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:01.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:01.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:01.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:02.148 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:02.147+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:02.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:02.190+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:03.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:03.123+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:03.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:03.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:04.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:04.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:04.154 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:04.153+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:05.119 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:05.119+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:05.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:05.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:06.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:06.163+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:06.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:06.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:06.803 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-29T21:30:06.804 INFO:tasks.ceph:Scrubbing osd.1 2023-09-29T21:30:06.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-29T21:30:07.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:07.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:07.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:07.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:08.072 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:08.073+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:08.252 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:08.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:09.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:09.097+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:09.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:09.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:10.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:10.117+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:10.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:10.308+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:11.076 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:11.077+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:11.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:11.303+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:12.110 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:12.110+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:12.279 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:12.280+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:13.088 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:13.088+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:13.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:13.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:14.095 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:14.095+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:14.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:14.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:15.131 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:15.131+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:15.259 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:15.259+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:16.112 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:16.112+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:16.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:16.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:17.158 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:17.158+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:17.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:17.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:18.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:18.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:18.240 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:18.241+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:19.195 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:19.196+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:19.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:19.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:20.177 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:20.177+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:20.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:20.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:21.216 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:21.216+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:21.242 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:21.242+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:22.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:22.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:22.259 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:22.259+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:23.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:23.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:23.217 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:23.217+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:24.172 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:24.172+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:24.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:24.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:25.176 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:25.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:25.212 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:25.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:26.176 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:26.176+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:26.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:26.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:27.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:27.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:27.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:27.207+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:28.159 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:28.158+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:28.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:28.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:29.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:29.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:29.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:29.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:30.096 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:30.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:30.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:30.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:31.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:31.098+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:31.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:31.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:32.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:32.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:32.126 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:32.126+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:33.090 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:33.090+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:33.115 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:33.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:34.082 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:34.082+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:34.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:34.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:35.049 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:35.049+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:35.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:35.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:36.022 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:36.022+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:36.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:36.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:36.993 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:36.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:37.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:37.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:38.001 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:38.001+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:38.099 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:38.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:39.010 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:39.011+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:39.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:39.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:39.971 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:39.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:40.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:40.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:40.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:40.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:41.070 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:41.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:41.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:41.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:42.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:42.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:42.897 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:42.897+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:43.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:43.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:43.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:43.932+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:44.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:44.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:44.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:44.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:45.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:45.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:45.943 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:45.943+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:46.073 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:46.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:46.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:46.937+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:47.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:47.110+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:47.978 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:47.978+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:48.130 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:48.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:48.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:48.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:49.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:49.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:50.026 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:50.026+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:50.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:50.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:51.064 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:51.064+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:51.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:51.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:52.093 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:52.094+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:52.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:52.113+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:53.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:53.055+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:53.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:53.162+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:54.042 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:54.043+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:54.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:54.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:55.062 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:55.063+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:55.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:55.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:56.068 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:56.068+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:56.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:56.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:57.118 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:57.118+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:57.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:57.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:58.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:58.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:58.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:58.151+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:59.120 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:30:59.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:30:59.124 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:30:59.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:00.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:00.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:00.092 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:00.092+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:01.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:01.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:01.125 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:01.124+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:02.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:02.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:02.148 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:02.147+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:03.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:03.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:03.135 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:03.135+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:04.071 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:04.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:04.087 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:04.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:05.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:05.055+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:05.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:05.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:06.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:06.012+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:06.077 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:06.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:06.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:06.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:07.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:07.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:07.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:07.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:08.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:08.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:09.007 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:09.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:09.013 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:09.013+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:09.973 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:09.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:10.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:10.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:10.957 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:10.956+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:11.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:11.014+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:11.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:11.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:12.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:12.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:13.016 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:13.015+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:13.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:13.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:13.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:13.976+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:14.072 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:14.071+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:14.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:14.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:15.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:15.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:15.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:15.936+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:16.143 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:16.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:16.909 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:16.908+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:17.097 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:17.096+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:17.873 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:17.872+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:18.064 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:18.063+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:18.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:18.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:19.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:19.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:19.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:19.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:20.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:20.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:20.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:20.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:21.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:21.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:21.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:21.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:22.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:22.027+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:22.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:22.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:23.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:23.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:23.786 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:23.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:24.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:24.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:24.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:24.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:25.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:25.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:25.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:25.826+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:26.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:26.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:26.830 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:26.829+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:27.045 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:27.044+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:27.816 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:27.815+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:28.034 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:28.033+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:28.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:28.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:29.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:29.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:29.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:29.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:30.063 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:30.062+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:30.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:30.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:31.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:31.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:31.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:31.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:32.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:32.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:32.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:32.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:33.133 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:33.132+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:33.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:33.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:34.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:34.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:34.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:34.753+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:35.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:35.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:35.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:35.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:36.128 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:36.127+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:36.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:36.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:37.140 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:37.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:37.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:37.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:38.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:38.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:38.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:38.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:39.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:39.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:39.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:39.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:40.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:40.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:40.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:40.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:41.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:41.092+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:41.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:41.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:42.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:42.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:42.736 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:42.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:43.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:43.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:43.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:43.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:44.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:44.157+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:44.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:44.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:45.135 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:45.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:45.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:45.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:46.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:46.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:46.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:46.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:47.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:47.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:47.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:47.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:48.095 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:48.094+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:48.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:48.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:49.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:49.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:49.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:49.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:50.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:50.059+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:50.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:50.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:51.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:51.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:51.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:51.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:52.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:52.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:52.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:52.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:52.988 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:52.988+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:53.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:53.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:53.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:53.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:54.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:54.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:54.976 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:54.976+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:55.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:55.598+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:56.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:56.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:56.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:56.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:57.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:57.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:57.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:57.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:58.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:58.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:58.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:58.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:59.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:31:59.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:31:59.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:31:59.542+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:00.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:00.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:00.507 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:00.507+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:01.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:01.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:01.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:01.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:02.167 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:02.166+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:02.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:02.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:03.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:03.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:03.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:03.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:04.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:04.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:04.531 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:04.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:05.173 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:05.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:05.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:05.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:06.146 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:06.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:06.440 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:06.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:06.833 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-29T21:32:06.834 INFO:tasks.ceph:Scrubbing osd.2 2023-09-29T21:32:06.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-29T21:32:06.976 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-09-29T21:32:06.976 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-29T21:32:06.977 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-09-29T21:32:06.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-29T21:32:07.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:07.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 2 to deep-scrub 2023-09-29T21:32:07.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:32:07.484 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:07.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:07.644 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:32:07.646 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:32:07.657 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5772,"stamp":"2023-09-29T21:32:07.231232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34871,"ondisk_log_size":34871,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":784268,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":191424,"kb_avail":282331252,"statfs":{"total":289910292480,"available":289107202048,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":196018176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001830"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"159'2384","reported_seq":3269,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074719+0000","last_change":"2023-09-29T18:30:06.801366+0000","last_active":"2023-09-29T19:27:48.074719+0000","last_peered":"2023-09-29T19:27:48.074719+0000","last_clean":"2023-09-29T19:27:48.074719+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T19:27:48.074719+0000","last_undegraded":"2023-09-29T19:27:48.074719+0000","last_fullsized":"2023-09-29T19:27:48.074719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2384,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:48:56.919234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"159'1917","reported_seq":2794,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.074758+0000","last_change":"2023-09-29T18:30:06.801380+0000","last_active":"2023-09-29T19:27:48.074758+0000","last_peered":"2023-09-29T19:27:48.074758+0000","last_clean":"2023-09-29T19:27:48.074758+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T19:27:48.074758+0000","last_undegraded":"2023-09-29T19:27:48.074758+0000","last_fullsized":"2023-09-29T19:27:48.074758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1917,"ondisk_log_size":1917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:07:26.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16450,"ondisk_log_size":16450,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740620,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246288,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":94125552,"statfs":{"total":96636764160,"available":96384565248,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20499999999999999,"15min":0.217},"min":{"1min":0.158,"5min":0.158,"15min":0.156},"max":{"1min":0.26700000000000002,"5min":0.27900000000000003,"15min":0.48999999999999999},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.28000000000000003,"5min":0.30599999999999999,"15min":0.47599999999999998},"last":0.188}]},{"osd":1,"last update":"Fri Sep 29 21:31:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.224,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.28499999999999998,"5min":0.314,"15min":0.48399999999999999},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.29799999999999999,"5min":0.313,"15min":0.50700000000000001},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740620,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.182,"5min":0.191,"15min":0.20300000000000001},"min":{"1min":0.152,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.252,"5min":0.36399999999999999,"15min":0.42499999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25800000000000001,"15min":0.27000000000000002},"min":{"1min":0.223,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.29199999999999998,"5min":0.42999999999999999,"15min":0.502},"last":0.27100000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22,"15min":0.23300000000000001},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.16300000000000001},"max":{"1min":0.28100000000000003,"5min":0.378,"15min":0.43099999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23899999999999999,"15min":0.252},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.17799999999999999},"max":{"1min":0.28100000000000003,"5min":0.39900000000000002,"15min":0.49199999999999999},"last":0.22600000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740618,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287796,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":60864,"kb_avail":94084044,"statfs":{"total":96636764160,"available":96342061056,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62324736},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.20699999999999999},"min":{"1min":0.156,"5min":0.14499999999999999,"15min":0.13900000000000001},"max":{"1min":0.26500000000000001,"5min":0.317,"15min":0.41299999999999998},"last":0.14799999999999999},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.23699999999999999},"min":{"1min":0.189,"5min":0.16900000000000001,"15min":0.16},"max":{"1min":0.26800000000000002,"5min":0.34399999999999997,"15min":0.44500000000000001},"last":0.21099999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20999999999999999,"15min":0.22500000000000001},"min":{"1min":0.17299999999999999,"5min":0.158,"15min":0.158},"max":{"1min":0.248,"5min":0.35399999999999998,"15min":0.44700000000000001},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.254,"5min":0.26100000000000001,"15min":0.27500000000000002},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.20499999999999999},"max":{"1min":0.29399999999999998,"5min":0.39000000000000001,"15min":0.497},"last":0.24299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:32:07.661 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.661 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.661 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.661 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.661 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.662 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.663 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.663 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:07.663 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:32:08.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:08.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:08.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:08.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:09.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:09.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:09.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:09.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:10.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:10.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:10.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:10.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:11.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:11.137+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:11.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:11.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:12.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:12.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:12.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:12.484+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:13.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:13.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:13.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:13.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:14.163 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:14.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:14.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:14.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:15.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:15.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:15.494 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:15.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:16.162 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:16.163+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:16.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:16.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:17.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:17.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:17.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:17.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:18.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:18.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:18.587 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:18.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:19.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:19.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:19.592 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:19.592+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:20.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:20.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:20.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:20.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:21.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:21.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:21.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:21.663+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:22.198 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:22.199+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:22.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:22.636+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:23.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:23.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:23.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:23.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:24.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:24.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:24.600 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:24.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:25.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:25.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:25.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:25.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:26.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:26.144+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:26.597 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:26.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:27.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:27.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:27.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:27.621+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:27.665 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:32:27.994 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:32:27.995 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:32:28.006 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5782,"stamp":"2023-09-29T21:32:27.234285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001775"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740624,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20499999999999999,"15min":0.217},"min":{"1min":0.158,"5min":0.158,"15min":0.156},"max":{"1min":0.26700000000000002,"5min":0.27900000000000003,"15min":0.48999999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.28000000000000003,"5min":0.30599999999999999,"15min":0.47599999999999998},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:31:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.224,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.28499999999999998,"5min":0.314,"15min":0.48399999999999999},"last":0.214},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.29799999999999999,"5min":0.313,"15min":0.50700000000000001},"last":0.27300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740624,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.182,"5min":0.191,"15min":0.20300000000000001},"min":{"1min":0.152,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.252,"5min":0.36399999999999999,"15min":0.42499999999999999},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25800000000000001,"15min":0.27000000000000002},"min":{"1min":0.223,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.29199999999999998,"5min":0.42999999999999999,"15min":0.502},"last":0.23400000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22,"15min":0.23300000000000001},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.16300000000000001},"max":{"1min":0.28100000000000003,"5min":0.378,"15min":0.43099999999999999},"last":0.221},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23899999999999999,"15min":0.252},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.17799999999999999},"max":{"1min":0.28100000000000003,"5min":0.39900000000000002,"15min":0.49199999999999999},"last":0.19800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740622,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.19,"5min":0.19500000000000001,"15min":0.20499999999999999},"min":{"1min":0.114,"5min":0.114,"15min":0.114},"max":{"1min":0.27000000000000002,"5min":0.317,"15min":0.41099999999999998},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23599999999999999},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.30499999999999999,"5min":0.34399999999999997,"15min":0.443},"last":0.21199999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.223},"min":{"1min":0.16400000000000001,"5min":0.158,"15min":0.158},"max":{"1min":0.29099999999999998,"5min":0.35399999999999998,"15min":0.44700000000000001},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.27300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.34200000000000003,"5min":0.39000000000000001,"15min":0.497},"last":0.22600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:32:28.008 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.008 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.008 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.008 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.009 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.009 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.009 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.009 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.009 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.010 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.010 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:28.010 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:32:28.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:28.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:28.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:28.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:29.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:29.241+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:29.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:29.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:30.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:30.246+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:30.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:30.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:31.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:31.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:31.661 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:31.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:32.284 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:32.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:32.658 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:32.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:33.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:33.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:33.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:33.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:34.327 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:34.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:34.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:34.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:35.316 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:35.316+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:35.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:35.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:36.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:36.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:36.769 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:36.769+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:37.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:37.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:37.819 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:37.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:38.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:38.384+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:38.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:38.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:39.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:39.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:39.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:39.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:40.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:40.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:40.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:40.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:41.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:41.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:41.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:41.791+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:42.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:42.347+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:42.784 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:42.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:43.363 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:43.363+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:43.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:43.773+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:44.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:44.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:44.799 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:44.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:45.374 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:45.374+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:45.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:45.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:46.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:46.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:46.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:46.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:47.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:47.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:47.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:47.747+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:48.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:32:48.327 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:32:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:32:48.339 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5792,"stamp":"2023-09-29T21:32:47.237353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001935"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740628,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20499999999999999,"15min":0.217},"min":{"1min":0.158,"5min":0.158,"15min":0.156},"max":{"1min":0.26700000000000002,"5min":0.27900000000000003,"15min":0.48999999999999999},"last":0.189},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.28000000000000003,"5min":0.30599999999999999,"15min":0.47599999999999998},"last":0.219}]},{"osd":1,"last update":"Fri Sep 29 21:31:51 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.224,"15min":0.23599999999999999},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.28499999999999998,"5min":0.314,"15min":0.48399999999999999},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.29799999999999999,"5min":0.313,"15min":0.50700000000000001},"last":0.26000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740628,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.189,"15min":0.20399999999999999},"min":{"1min":0.14599999999999999,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.42499999999999999},"last":0.192},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.27100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.502},"last":0.28499999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23300000000000001},"min":{"1min":0.187,"5min":0.17299999999999999,"15min":0.16700000000000001},"max":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.43099999999999999},"last":0.222},{"interface":"front","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.252},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.33000000000000002,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.255}]}]},{"osd":1,"up_from":8,"seq":34359740626,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.19,"5min":0.19500000000000001,"15min":0.20499999999999999},"min":{"1min":0.114,"5min":0.114,"15min":0.114},"max":{"1min":0.27000000000000002,"5min":0.317,"15min":0.41099999999999998},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23599999999999999},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.30499999999999999,"5min":0.34399999999999997,"15min":0.443},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.223},"min":{"1min":0.16400000000000001,"5min":0.158,"15min":0.158},"max":{"1min":0.29099999999999998,"5min":0.35399999999999998,"15min":0.44700000000000001},"last":0.224},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.27300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.34200000000000003,"5min":0.39000000000000001,"15min":0.497},"last":0.29299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:32:48.341 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.341 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.341 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.342 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.342 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.342 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.342 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.342 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.343 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.343 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.343 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:32:48.343 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:32:48.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:48.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:48.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:48.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:49.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:49.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:49.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:49.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:50.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:50.425+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:50.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:50.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:51.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:51.425+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:51.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:51.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:52.435 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:52.435+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:52.752 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:52.752+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:53.485 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:53.485+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:53.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:53.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:54.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:54.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:54.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:54.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:55.531 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:55.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:55.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:55.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:56.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:56.532+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:56.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:56.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:57.550 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:57.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:57.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:57.789+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:58.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:58.506+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:58.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:58.784+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:59.546 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:32:59.546+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:32:59.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:32:59.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:00.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:00.537+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:00.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:00.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:01.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:01.529+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:01.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:01.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:02.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:02.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:02.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:02.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:03.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:03.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:03.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:03.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:04.592 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:04.591+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:04.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:04.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:05.575 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:05.574+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:05.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:05.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:06.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:06.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:06.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:06.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:07.576 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:07.576+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:07.751 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:07.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:08.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:33:08.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:08.620+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:08.687 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:33:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:33:08.699 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5802,"stamp":"2023-09-29T21:33:07.240600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001860"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740632,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.20599999999999999,"15min":0.215},"min":{"1min":0.16,"5min":0.158,"15min":0.156},"max":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.48999999999999999},"last":0.185},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23599999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.47599999999999998},"last":0.215}]},{"osd":1,"last update":"Fri Sep 29 21:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.23699999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.251,"15min":0.26100000000000001},"min":{"1min":0.22,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.313,"5min":0.313,"15min":0.50700000000000001},"last":0.25600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740632,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.189,"15min":0.20399999999999999},"min":{"1min":0.14599999999999999,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.42499999999999999},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.27100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.502},"last":0.27100000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23300000000000001},"min":{"1min":0.187,"5min":0.17299999999999999,"15min":0.16700000000000001},"max":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.43099999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.252},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.33000000000000002,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.23599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740630,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.19,"5min":0.19500000000000001,"15min":0.20499999999999999},"min":{"1min":0.114,"5min":0.114,"15min":0.114},"max":{"1min":0.27000000000000002,"5min":0.317,"15min":0.41099999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.22700000000000001,"5min":0.22500000000000001,"15min":0.23599999999999999},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.30499999999999999,"5min":0.34399999999999997,"15min":0.443},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.223},"min":{"1min":0.16400000000000001,"5min":0.158,"15min":0.158},"max":{"1min":0.29099999999999998,"5min":0.35399999999999998,"15min":0.44700000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.27300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.34200000000000003,"5min":0.39000000000000001,"15min":0.497},"last":0.35699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:33:08.701 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.701 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.701 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.702 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.702 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.702 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.702 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.702 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.703 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.703 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.703 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:08.703 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:33:08.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:08.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:09.615 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:09.613+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:09.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:09.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:10.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:10.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:10.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:10.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:11.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:11.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:11.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:11.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:12.606 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:12.605+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:12.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:12.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:13.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:13.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:13.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:13.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:14.526 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:14.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:14.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:14.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:15.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:15.512+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:15.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:15.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:16.475 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:16.474+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:16.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:16.606+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:17.427 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:17.426+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:17.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:17.619+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:18.410 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:18.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:18.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:18.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:19.379 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:19.378+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:19.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:19.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:20.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:20.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:20.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:20.680+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:21.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:21.420+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:21.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:21.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:22.377 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:22.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:22.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:22.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:23.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:23.385+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:23.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:23.792+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:24.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:24.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:24.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:24.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:25.389 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:25.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:25.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:25.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:26.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:26.436+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:26.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:26.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:27.407 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:27.406+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:27.732 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:27.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:28.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:28.454+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:28.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:33:28.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:28.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:29.024 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:33:29.026 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:33:29.037 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5812,"stamp":"2023-09-29T21:33:27.243789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001817"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740636,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.20599999999999999,"15min":0.215},"min":{"1min":0.16,"5min":0.158,"15min":0.156},"max":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.48999999999999999},"last":0.223},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23599999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.47599999999999998},"last":0.187}]},{"osd":1,"last update":"Fri Sep 29 21:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.23699999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.251,"15min":0.26100000000000001},"min":{"1min":0.22,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.313,"5min":0.313,"15min":0.50700000000000001},"last":0.23699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740636,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.189,"15min":0.20399999999999999},"min":{"1min":0.14599999999999999,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.42499999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.27100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.502},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23300000000000001},"min":{"1min":0.187,"5min":0.17299999999999999,"15min":0.16700000000000001},"max":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.43099999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.252},"min":{"1min":0.19800000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.33000000000000002,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.27200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740634,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.19800000000000001,"15min":0.20499999999999999},"min":{"1min":0.13700000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.41099999999999998},"last":0.151},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.16200000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.443},"last":0.19}]},{"osd":2,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.214,"15min":0.223},"min":{"1min":0.17799999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.44700000000000001},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.23200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:33:29.039 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.039 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.040 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.040 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.040 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.040 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.040 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.041 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.041 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.041 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.041 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:29.041 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:33:29.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:29.482+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:29.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:29.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:30.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:30.451+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:30.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:30.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:31.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:31.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:31.811 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:31.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:32.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:32.506+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:32.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:32.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:33.523 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:33.522+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:33.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:33.861+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:34.521 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:34.520+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:34.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:34.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:35.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:35.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:35.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:35.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:36.553 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:36.552+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:36.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:36.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:37.506 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:37.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:37.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:37.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:38.552 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:38.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:38.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:38.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:39.587 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:39.585+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:39.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:39.833+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:40.565 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:40.564+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:40.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:40.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:41.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:41.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:41.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:41.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:42.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:42.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:42.797 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:42.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:43.561 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:43.560+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:43.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:43.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:44.534 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:44.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:44.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:44.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:45.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:45.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:45.746 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:45.746+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:46.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:46.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:46.772 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:46.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:47.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:47.610+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:47.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:47.730+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:48.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:48.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:48.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:48.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:49.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:33:49.358 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:33:49.360 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:33:49.373 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5823,"stamp":"2023-09-29T21:33:49.247051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001810"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740641,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.20599999999999999,"15min":0.215},"min":{"1min":0.16,"5min":0.158,"15min":0.156},"max":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.48999999999999999},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23599999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.47599999999999998},"last":0.23699999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.23699999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.251},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.251,"15min":0.26100000000000001},"min":{"1min":0.22,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.313,"5min":0.313,"15min":0.50700000000000001},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740641,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.193,"15min":0.20200000000000001},"min":{"1min":0.159,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.42499999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.219,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.502},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23100000000000001},"min":{"1min":0.191,"5min":0.183,"15min":0.16700000000000001},"max":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.42299999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.252,"5min":0.245,"15min":0.25},"min":{"1min":0.20000000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.377,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.318}]}]},{"osd":1,"up_from":8,"seq":34359740639,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.19800000000000001,"15min":0.20499999999999999},"min":{"1min":0.13700000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.41099999999999998},"last":0.17299999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.16200000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.443},"last":0.19900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.214,"15min":0.223},"min":{"1min":0.17799999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.44700000000000001},"last":0.188},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:33:49.374 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.374 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.375 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.375 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.375 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.375 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:33:49.376 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:33:49.649 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:49.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:49.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:49.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:50.620 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:50.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:50.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:50.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:51.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:51.581+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:51.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:51.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:52.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:52.543+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:52.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:52.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:53.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:53.556+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:53.572 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:53.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:54.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:54.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:54.563 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:54.563+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:55.564 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:55.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:55.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:55.567+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:56.529 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:56.528+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:56.558 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:56.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:57.537 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:57.536+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:57.539 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:57.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:58.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:58.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:58.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:58.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:59.533 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:33:59.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:33:59.607 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:33:59.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:00.504 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:00.505+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:00.621 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:00.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:01.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:01.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:01.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:01.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:02.503 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:02.503+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:02.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:02.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:03.507 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:03.507+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:03.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:03.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:04.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:04.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:04.762 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:04.762+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:05.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:05.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:05.812 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:05.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:06.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:06.555+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:06.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:06.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:07.593 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:07.593+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:07.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:07.796+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:08.643 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:08.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:08.814 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:08.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:09.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:34:09.607 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:09.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:09.731 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:34:09.733 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:34:09.744 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5833,"stamp":"2023-09-29T21:34:09.250092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001767"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740645,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.21099999999999999,"15min":0.216},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.20799999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.191,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.313,"5min":1.044,"15min":1.044},"last":0.222},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.26200000000000001},"min":{"1min":0.217,"5min":0.187,"15min":0.17799999999999999},"max":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.50700000000000001},"last":0.25}]}]},{"osd":0,"up_from":8,"seq":34359740645,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.193,"15min":0.20200000000000001},"min":{"1min":0.159,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.42499999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.219,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.502},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23100000000000001},"min":{"1min":0.191,"5min":0.183,"15min":0.16700000000000001},"max":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.42299999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.245,"15min":0.25},"min":{"1min":0.20000000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.377,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.23599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740643,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.19800000000000001,"15min":0.20499999999999999},"min":{"1min":0.13700000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.41099999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23599999999999999},"min":{"1min":0.16200000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.443},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.214,"15min":0.223},"min":{"1min":0.17799999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.44700000000000001},"last":0.19},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.27300000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.25700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:34:09.746 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.746 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.746 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.746 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.747 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.747 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.747 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.747 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.747 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.748 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.748 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:09.748 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:34:09.796 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:09.797+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:10.631 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:10.631+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:10.806 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:10.807+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:11.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:11.634+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:11.777 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:11.777+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:12.635 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:12.635+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:12.805 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:12.805+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:13.611 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:13.611+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:13.775 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:13.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:14.644 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:14.644+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:14.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:14.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:15.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:15.692+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:15.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:15.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:16.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:16.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:16.780 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:16.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:17.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:17.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:17.735 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:17.735+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:18.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:18.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:18.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:18.725+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:19.690 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:19.690+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:19.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:19.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:20.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:20.696+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:20.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:20.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:21.704 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:21.704+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:21.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:21.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:22.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:22.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:22.733 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:22.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:23.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:23.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:23.749 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:23.749+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:24.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:24.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:24.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:24.772+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:25.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:25.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:25.723 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:25.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:26.622 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:26.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:26.702 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:26.702+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:27.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:27.640+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:27.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:27.695+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:28.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:28.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:28.750 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:28.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:29.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:29.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:29.749 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:34:29.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:29.758+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:30.076 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:34:30.079 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:34:30.090 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5843,"stamp":"2023-09-29T21:34:29.253070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34997,"ondisk_log_size":34997,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":788684,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":195840,"kb_avail":282326836,"statfs":{"total":289910292480,"available":289102680064,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200540160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001752"},"pg_stats":[{"pgid":"2.7","version":"159'1944","reported_seq":2966,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075309+0000","last_change":"2023-09-29T18:30:06.808055+0000","last_active":"2023-09-29T19:27:48.075309+0000","last_peered":"2023-09-29T19:27:48.075309+0000","last_clean":"2023-09-29T19:27:48.075309+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T19:27:48.075309+0000","last_undegraded":"2023-09-29T19:27:48.075309+0000","last_fullsized":"2023-09-29T19:27:48.075309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1944,"ondisk_log_size":1944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T21:48:49.468743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2985,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075282+0000","last_change":"2023-09-29T18:30:06.807974+0000","last_active":"2023-09-29T19:27:48.075282+0000","last_peered":"2023-09-29T19:27:48.075282+0000","last_clean":"2023-09-29T19:27:48.075282+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T19:27:48.075282+0000","last_undegraded":"2023-09-29T19:27:48.075282+0000","last_fullsized":"2023-09-29T19:27:48.075282+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:39:11.001107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"159'1935","reported_seq":2859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075192+0000","last_change":"2023-09-29T18:30:06.808179+0000","last_active":"2023-09-29T19:27:48.075192+0000","last_peered":"2023-09-29T19:27:48.075192+0000","last_clean":"2023-09-29T19:27:48.075192+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T19:27:48.075192+0000","last_undegraded":"2023-09-29T19:27:48.075192+0000","last_fullsized":"2023-09-29T19:27:48.075192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1935,"ondisk_log_size":1935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:16.522182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"159'4329","reported_seq":9060,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075846+0000","last_change":"2023-09-29T18:30:06.799189+0000","last_active":"2023-09-29T19:27:48.075846+0000","last_peered":"2023-09-29T19:27:48.075846+0000","last_clean":"2023-09-29T19:27:48.075846+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T19:27:48.075846+0000","last_undegraded":"2023-09-29T19:27:48.075846+0000","last_fullsized":"2023-09-29T19:27:48.075846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4329,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T22:23:27.165583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12735,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075231+0000","last_change":"2023-09-29T18:30:06.809322+0000","last_active":"2023-09-29T19:27:48.075231+0000","last_peered":"2023-09-29T19:27:48.075231+0000","last_clean":"2023-09-29T19:27:48.075231+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T19:27:48.075231+0000","last_undegraded":"2023-09-29T19:27:48.075231+0000","last_fullsized":"2023-09-29T19:27:48.075231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T18:57:21.584268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075069+0000","last_change":"2023-09-29T18:24:59.530749+0000","last_active":"2023-09-29T19:27:48.075069+0000","last_peered":"2023-09-29T19:27:48.075069+0000","last_clean":"2023-09-29T19:27:48.075069+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T19:27:48.075069+0000","last_undegraded":"2023-09-29T19:27:48.075069+0000","last_fullsized":"2023-09-29T19:27:48.075069+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:43:00.595350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3470,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075120+0000","last_change":"2023-09-29T18:30:06.809087+0000","last_active":"2023-09-29T19:27:48.075120+0000","last_peered":"2023-09-29T19:27:48.075120+0000","last_clean":"2023-09-29T19:27:48.075120+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T19:27:48.075120+0000","last_undegraded":"2023-09-29T19:27:48.075120+0000","last_fullsized":"2023-09-29T19:27:48.075120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T20:12:55.098504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10811,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075099+0000","last_change":"2023-09-29T18:30:06.809049+0000","last_active":"2023-09-29T19:27:48.075099+0000","last_peered":"2023-09-29T19:27:48.075099+0000","last_clean":"2023-09-29T19:27:48.075099+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T19:27:48.075099+0000","last_undegraded":"2023-09-29T19:27:48.075099+0000","last_fullsized":"2023-09-29T19:27:48.075099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:05:53.471891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12035,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075859+0000","last_change":"2023-09-29T18:30:06.801767+0000","last_active":"2023-09-29T19:27:48.075859+0000","last_peered":"2023-09-29T19:27:48.075859+0000","last_clean":"2023-09-29T19:27:48.075859+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T19:27:48.075859+0000","last_undegraded":"2023-09-29T19:27:48.075859+0000","last_fullsized":"2023-09-29T19:27:48.075859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:17:51.328669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"159'1719","reported_seq":2799,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075856+0000","last_change":"2023-09-29T18:30:06.798465+0000","last_active":"2023-09-29T19:27:48.075856+0000","last_peered":"2023-09-29T19:27:48.075856+0000","last_clean":"2023-09-29T19:27:48.075856+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T19:27:48.075856+0000","last_undegraded":"2023-09-29T19:27:48.075856+0000","last_fullsized":"2023-09-29T19:27:48.075856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":1719,"ondisk_log_size":1719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:43.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3166,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-09-29T19:27:48.075203+0000","last_change":"2023-09-29T18:30:06.808369+0000","last_active":"2023-09-29T19:27:48.075203+0000","last_peered":"2023-09-29T19:27:48.075203+0000","last_clean":"2023-09-29T19:27:48.075203+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T19:27:48.075203+0000","last_undegraded":"2023-09-29T19:27:48.075203+0000","last_fullsized":"2023-09-29T19:27:48.075203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:45:45.725726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18324,"ondisk_log_size":18324,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16576,"ondisk_log_size":16576,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740649,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.21099999999999999,"15min":0.216},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.191},{"interface":"front","average":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.191,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.313,"5min":1.044,"15min":1.044},"last":0.219},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.26200000000000001},"min":{"1min":0.217,"5min":0.187,"15min":0.17799999999999999},"max":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.50700000000000001},"last":0.27700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740649,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250184,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63744,"kb_avail":94121656,"statfs":{"total":96636764160,"available":96380575744,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65273856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.193,"15min":0.20200000000000001},"min":{"1min":0.159,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.42499999999999999},"last":0.14399999999999999},{"interface":"front","average":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.219,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.502},"last":0.20799999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.222,"15min":0.23100000000000001},"min":{"1min":0.191,"5min":0.183,"15min":0.16700000000000001},"max":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.42299999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.245,"15min":0.25},"min":{"1min":0.20000000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.377,"5min":0.39100000000000001,"15min":0.49199999999999999},"last":0.24399999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740647,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292020,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65088,"kb_avail":94079820,"statfs":{"total":96636764160,"available":96337735680,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66650112},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20000000000000001,"15min":0.20399999999999999},"min":{"1min":0.16200000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.19900000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.443},"last":0.20899999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.216,"15min":0.223},"min":{"1min":0.17599999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.188},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.27300000000000002},"min":{"1min":0.217,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.45700000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.23100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:34:30.091 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.091 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.092 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.092 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.092 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.092 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.092 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.093 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.093 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.093 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.093 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:30.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-09-29T21:34:30.446 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-09-29T21:34:30.458 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:34:30.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:30.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:30.777 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:34:30.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:30.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:30.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-09-29T21:34:31.165 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-09-29T21:34:31.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-09-29T21:34:31.544 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-09-29T21:34:31.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:34:31.605 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:31.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:31.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:31.763+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:31.882 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:34:31.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:34:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:34:32.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:34:32.611 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:34:32.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:34:32.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:32.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:32.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:32.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:33.029 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:34:33.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:34:33.397 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:34:33.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-09-29T21:34:33.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:33.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:33.734 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-09-29T21:34:33.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:34:33.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:33.797+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:34.158 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:34:34.170 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:34:34.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:34.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:34.828 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:34.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:35.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:35.521+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:35.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:35.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:36.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:36.495+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:36.834 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:36.835+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:37.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:37.468+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:37.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:37.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:38.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:38.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:38.905 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:38.905+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:39.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:39.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:39.941 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:39.941+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:40.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:40.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:40.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:40.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:41.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:41.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:42.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:42.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:42.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:42.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:42.963 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:42.963+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:43.497 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:43.497+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:43.972 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:43.972+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:44.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:44.491+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:45.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:45.001+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:45.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:45.493+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:46.036 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:46.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:46.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:46.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:47.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:47.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:47.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:47.534+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:48.010 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:48.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:48.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:48.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:49.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:49.032+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:49.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:49.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:50.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:50.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:50.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:50.504+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:51.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:51.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:51.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:51.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:52.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:52.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:52.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:52.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:53.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:53.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:53.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:53.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:54.136 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:54.135+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:54.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:34:54.462 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:54.461+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:54.486 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:34:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:34:54.499 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5855,"stamp":"2023-09-29T21:34:53.257120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001980"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2989,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:39.942018+0000","last_change":"2023-09-29T21:34:39.942018+0000","last_active":"2023-09-29T21:34:39.942018+0000","last_peered":"2023-09-29T21:34:39.942018+0000","last_clean":"2023-09-29T21:34:39.942018+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:34:39.942018+0000","last_undegraded":"2023-09-29T21:34:39.942018+0000","last_fullsized":"2023-09-29T21:34:39.942018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12739,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:46.036722+0000","last_change":"2023-09-29T21:34:46.036722+0000","last_active":"2023-09-29T21:34:46.036722+0000","last_peered":"2023-09-29T21:34:46.036722+0000","last_clean":"2023-09-29T21:34:46.036722+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:34:46.036722+0000","last_undegraded":"2023-09-29T21:34:46.036722+0000","last_fullsized":"2023-09-29T21:34:46.036722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":290,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.797723+0000","last_change":"2023-09-29T21:34:32.797723+0000","last_active":"2023-09-29T21:34:32.797723+0000","last_peered":"2023-09-29T21:34:32.797723+0000","last_clean":"2023-09-29T21:34:32.797723+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:34:32.797723+0000","last_undegraded":"2023-09-29T21:34:32.797723+0000","last_fullsized":"2023-09-29T21:34:32.797723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3474,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:37.859601+0000","last_change":"2023-09-29T21:34:37.859601+0000","last_active":"2023-09-29T21:34:37.859601+0000","last_peered":"2023-09-29T21:34:37.859601+0000","last_clean":"2023-09-29T21:34:37.859601+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:34:37.859601+0000","last_undegraded":"2023-09-29T21:34:37.859601+0000","last_fullsized":"2023-09-29T21:34:37.859601+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10815,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:42.005765+0000","last_change":"2023-09-29T21:34:42.005765+0000","last_active":"2023-09-29T21:34:42.005765+0000","last_peered":"2023-09-29T21:34:42.005765+0000","last_clean":"2023-09-29T21:34:42.005765+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:34:42.005765+0000","last_undegraded":"2023-09-29T21:34:42.005765+0000","last_fullsized":"2023-09-29T21:34:42.005765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3170,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:38.905942+0000","last_change":"2023-09-29T21:34:38.905942+0000","last_active":"2023-09-29T21:34:38.905942+0000","last_peered":"2023-09-29T21:34:38.905942+0000","last_clean":"2023-09-29T21:34:38.905942+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:34:38.905942+0000","last_undegraded":"2023-09-29T21:34:38.905942+0000","last_fullsized":"2023-09-29T21:34:38.905942+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740654,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.21099999999999999,"15min":0.216},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.185},{"interface":"front","average":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23699999999999999},"min":{"1min":0.187,"5min":0.17399999999999999,"15min":0.16200000000000001},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.247}]},{"osd":1,"last update":"Fri Sep 29 21:33:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.191,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.313,"5min":1.044,"15min":1.044},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.26200000000000001},"min":{"1min":0.217,"5min":0.187,"15min":0.17799999999999999},"max":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.50700000000000001},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740654,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.19500000000000001,"15min":0.20200000000000001},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.183},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.27000000000000002},"min":{"1min":0.20799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22800000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16700000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.253,"15min":0.253},"min":{"1min":0.22700000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.49199999999999999},"last":0.29499999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740652,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20000000000000001,"15min":0.20399999999999999},"min":{"1min":0.16200000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.191},{"interface":"front","average":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.19900000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.443},"last":0.22600000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.216,"15min":0.223},"min":{"1min":0.17599999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.27300000000000002},"min":{"1min":0.217,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.45700000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.23799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:34:54.501 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.501 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.502 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.502 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.502 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.502 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.502 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:34:54.503 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:34:55.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:55.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:55.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:55.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:56.181 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:56.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:56.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:56.453+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:57.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:57.214+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:57.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:57.492+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:58.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:58.257+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:58.461 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:58.460+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:59.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:34:59.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:34:59.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:34:59.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:00.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:00.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:00.480 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:00.479+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:01.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:01.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:01.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:01.519+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:02.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:02.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:02.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:02.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:03.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:03.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:03.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:03.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:04.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:04.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:04.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:04.545+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:05.221 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:05.220+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:05.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:05.554+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:06.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:06.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:06.508 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:06.507+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:07.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:07.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:07.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:07.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:08.219 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:08.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:08.501 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:08.500+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:09.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:09.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:09.482 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:09.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:10.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:10.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:10.455 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:10.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:11.239 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:11.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:11.419 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:11.418+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:12.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:12.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:12.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:12.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:13.189 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:13.188+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:13.468 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:13.467+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:14.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:14.194+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:14.421 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:14.420+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:14.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:35:14.848 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:35:14.851 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:35:14.863 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5865,"stamp":"2023-09-29T21:35:13.260365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001859"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2992,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:08.219992+0000","last_change":"2023-09-29T21:35:08.219992+0000","last_active":"2023-09-29T21:35:08.219992+0000","last_peered":"2023-09-29T21:35:08.219992+0000","last_clean":"2023-09-29T21:35:08.219992+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:35:08.219992+0000","last_undegraded":"2023-09-29T21:35:08.219992+0000","last_fullsized":"2023-09-29T21:35:08.219992+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12739,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:46.036722+0000","last_change":"2023-09-29T21:34:46.036722+0000","last_active":"2023-09-29T21:34:46.036722+0000","last_peered":"2023-09-29T21:34:46.036722+0000","last_clean":"2023-09-29T21:34:46.036722+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:34:46.036722+0000","last_undegraded":"2023-09-29T21:34:46.036722+0000","last_fullsized":"2023-09-29T21:34:46.036722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":293,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:53.110684+0000","last_change":"2023-09-29T21:34:53.110684+0000","last_active":"2023-09-29T21:34:53.110684+0000","last_peered":"2023-09-29T21:34:53.110684+0000","last_clean":"2023-09-29T21:34:53.110684+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:34:53.110684+0000","last_undegraded":"2023-09-29T21:34:53.110684+0000","last_fullsized":"2023-09-29T21:34:53.110684+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3477,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:58.258527+0000","last_change":"2023-09-29T21:34:58.258527+0000","last_active":"2023-09-29T21:34:58.258527+0000","last_peered":"2023-09-29T21:34:58.258527+0000","last_clean":"2023-09-29T21:34:58.258527+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:34:58.258527+0000","last_undegraded":"2023-09-29T21:34:58.258527+0000","last_fullsized":"2023-09-29T21:34:58.258527+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10818,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:10.207313+0000","last_change":"2023-09-29T21:35:10.207313+0000","last_active":"2023-09-29T21:35:10.207313+0000","last_peered":"2023-09-29T21:35:10.207313+0000","last_clean":"2023-09-29T21:35:10.207313+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:35:10.207313+0000","last_undegraded":"2023-09-29T21:35:10.207313+0000","last_fullsized":"2023-09-29T21:35:10.207313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3173,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:04.263441+0000","last_change":"2023-09-29T21:35:04.263441+0000","last_active":"2023-09-29T21:35:04.263441+0000","last_peered":"2023-09-29T21:35:04.263441+0000","last_clean":"2023-09-29T21:35:04.263441+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:35:04.263441+0000","last_undegraded":"2023-09-29T21:35:04.263441+0000","last_fullsized":"2023-09-29T21:35:04.263441+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740658,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.214,"15min":0.217},"min":{"1min":0.161,"5min":0.154,"15min":0.154},"max":{"1min":0.36799999999999999,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23799999999999999},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17000000000000001},"max":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.186}]},{"osd":1,"last update":"Fri Sep 29 21:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24099999999999999},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.74099999999999999,"5min":1.044,"15min":1.044},"last":0.249},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.22600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740658,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.19500000000000001,"15min":0.20200000000000001},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.186},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.27000000000000002},"min":{"1min":0.20799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22800000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16700000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.253,"15min":0.253},"min":{"1min":0.22700000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.49199999999999999},"last":0.23699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20000000000000001,"15min":0.20399999999999999},"min":{"1min":0.16200000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.187},{"interface":"front","average":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23699999999999999},"min":{"1min":0.19900000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.443},"last":0.22500000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:34:21 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.216,"15min":0.223},"min":{"1min":0.17599999999999999,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.27300000000000002},"min":{"1min":0.217,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.45700000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.251}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:35:14.864 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.865 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.865 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.865 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.865 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.865 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.866 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:14.866 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:35:15.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:15.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:15.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:15.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:16.123 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:16.122+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:16.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:16.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:17.081 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:17.080+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:17.437 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:17.436+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:18.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:18.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:18.452 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:18.451+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:19.109 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:19.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:19.449 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:19.448+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:20.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:20.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:20.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:20.489+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:21.129 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:21.128+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:21.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:21.458+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:22.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:22.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:22.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:22.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:23.137 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:23.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:23.496 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:23.496+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:24.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:24.116+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:24.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:24.509+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:25.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:25.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:25.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:25.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:26.122 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:26.121+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:26.450 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:26.450+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:27.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:27.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:27.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:27.438+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:28.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:28.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:28.453 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:28.452+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:29.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:29.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:29.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:29.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:30.223 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:30.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:30.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:30.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:31.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:31.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:31.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:31.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:32.149 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:32.148+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:32.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:32.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:33.174 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:33.173+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:33.476 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:33.475+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:34.158 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:34.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:34.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:34.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:34.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:35:35.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:35.170+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:35.184 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:35:35.186 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:35:35.197 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5875,"stamp":"2023-09-29T21:35:33.263259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001758"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2995,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:22.113253+0000","last_change":"2023-09-29T21:35:22.113253+0000","last_active":"2023-09-29T21:35:22.113253+0000","last_peered":"2023-09-29T21:35:22.113253+0000","last_clean":"2023-09-29T21:35:22.113253+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:35:22.113253+0000","last_undegraded":"2023-09-29T21:35:22.113253+0000","last_fullsized":"2023-09-29T21:35:22.113253+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12745,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:31.178233+0000","last_change":"2023-09-29T21:35:31.178233+0000","last_active":"2023-09-29T21:35:31.178233+0000","last_peered":"2023-09-29T21:35:31.178233+0000","last_clean":"2023-09-29T21:35:31.178233+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:35:31.178233+0000","last_undegraded":"2023-09-29T21:35:31.178233+0000","last_fullsized":"2023-09-29T21:35:31.178233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":296,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:15.159574+0000","last_change":"2023-09-29T21:35:15.159574+0000","last_active":"2023-09-29T21:35:15.159574+0000","last_peered":"2023-09-29T21:35:15.159574+0000","last_clean":"2023-09-29T21:35:15.159574+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:35:15.159574+0000","last_undegraded":"2023-09-29T21:35:15.159574+0000","last_fullsized":"2023-09-29T21:35:15.159574+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3480,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:19.110281+0000","last_change":"2023-09-29T21:35:19.110281+0000","last_active":"2023-09-29T21:35:19.110281+0000","last_peered":"2023-09-29T21:35:19.110281+0000","last_clean":"2023-09-29T21:35:19.110281+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:35:19.110281+0000","last_undegraded":"2023-09-29T21:35:19.110281+0000","last_fullsized":"2023-09-29T21:35:19.110281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10821,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:25.085897+0000","last_change":"2023-09-29T21:35:25.085897+0000","last_active":"2023-09-29T21:35:25.085897+0000","last_peered":"2023-09-29T21:35:25.085897+0000","last_clean":"2023-09-29T21:35:25.085897+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:35:25.085897+0000","last_undegraded":"2023-09-29T21:35:25.085897+0000","last_fullsized":"2023-09-29T21:35:25.085897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3176,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:21.129625+0000","last_change":"2023-09-29T21:35:21.129625+0000","last_active":"2023-09-29T21:35:21.129625+0000","last_peered":"2023-09-29T21:35:21.129625+0000","last_clean":"2023-09-29T21:35:21.129625+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:35:21.129625+0000","last_undegraded":"2023-09-29T21:35:21.129625+0000","last_fullsized":"2023-09-29T21:35:21.129625+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740662,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.214,"15min":0.217},"min":{"1min":0.161,"5min":0.154,"15min":0.154},"max":{"1min":0.36799999999999999,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23799999999999999},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17000000000000001},"max":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.20399999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24099999999999999},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.74099999999999999,"5min":1.044,"15min":1.044},"last":0.22},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.23000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740662,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.19500000000000001,"15min":0.20200000000000001},"min":{"1min":0.14399999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.27000000000000002},"min":{"1min":0.20799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.311}]},{"osd":2,"last update":"Fri Sep 29 21:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22800000000000001,"15min":0.23200000000000001},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16700000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.253,"15min":0.253},"min":{"1min":0.22700000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.49199999999999999},"last":0.27900000000000003}]}]},{"osd":1,"up_from":8,"seq":34359740660,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.20699999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39400000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.16300000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35799999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.251}]},{"osd":2,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.221,"15min":0.223},"min":{"1min":0.188,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.371,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.221},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.27300000000000002,"15min":0.27400000000000002},"min":{"1min":0.23100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40699999999999997,"5min":0.47299999999999998,"15min":0.497},"last":0.26300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:35:35.199 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.199 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.199 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.199 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.199 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.200 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.200 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:35.200 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:35:35.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:35.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:36.127 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:36.126+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:36.598 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:36.597+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:37.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:37.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:37.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:37.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:38.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:38.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:38.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:38.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:39.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:39.084+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:39.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:39.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:40.043 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:40.042+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:40.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:40.683+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:41.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:41.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:41.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:41.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:42.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:42.034+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:42.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:42.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:43.023 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:43.022+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:43.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:43.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:44.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:44.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:44.722 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:44.722+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:45.051 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:45.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:45.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:45.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:46.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:46.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:46.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:46.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:47.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:47.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:47.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:47.787+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:48.103 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:48.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:48.783 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:48.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:49.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:49.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:49.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:49.772+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:50.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:50.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:50.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:50.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:51.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:51.109+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:51.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:51.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:52.107 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:52.107+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:52.865 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:52.864+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:53.061 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:53.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:53.843 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:53.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:54.086 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:54.086+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:54.836 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:54.836+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:55.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:55.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:55.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:35:55.524 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:35:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:35:55.537 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5886,"stamp":"2023-09-29T21:35:55.267085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002313"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2995,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:22.113253+0000","last_change":"2023-09-29T21:35:22.113253+0000","last_active":"2023-09-29T21:35:22.113253+0000","last_peered":"2023-09-29T21:35:22.113253+0000","last_clean":"2023-09-29T21:35:22.113253+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:35:22.113253+0000","last_undegraded":"2023-09-29T21:35:22.113253+0000","last_fullsized":"2023-09-29T21:35:22.113253+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12745,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:31.178233+0000","last_change":"2023-09-29T21:35:31.178233+0000","last_active":"2023-09-29T21:35:31.178233+0000","last_peered":"2023-09-29T21:35:31.178233+0000","last_clean":"2023-09-29T21:35:31.178233+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:35:31.178233+0000","last_undegraded":"2023-09-29T21:35:31.178233+0000","last_fullsized":"2023-09-29T21:35:31.178233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":299,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:37.104065+0000","last_change":"2023-09-29T21:35:37.104065+0000","last_active":"2023-09-29T21:35:37.104065+0000","last_peered":"2023-09-29T21:35:37.104065+0000","last_clean":"2023-09-29T21:35:37.104065+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:35:37.104065+0000","last_undegraded":"2023-09-29T21:35:37.104065+0000","last_fullsized":"2023-09-29T21:35:37.104065+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3483,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:51.110626+0000","last_change":"2023-09-29T21:35:51.110626+0000","last_active":"2023-09-29T21:35:51.110626+0000","last_peered":"2023-09-29T21:35:51.110626+0000","last_clean":"2023-09-29T21:35:51.110626+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:35:51.110626+0000","last_undegraded":"2023-09-29T21:35:51.110626+0000","last_fullsized":"2023-09-29T21:35:51.110626+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10821,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:25.085897+0000","last_change":"2023-09-29T21:35:25.085897+0000","last_active":"2023-09-29T21:35:25.085897+0000","last_peered":"2023-09-29T21:35:25.085897+0000","last_clean":"2023-09-29T21:35:25.085897+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:35:25.085897+0000","last_undegraded":"2023-09-29T21:35:25.085897+0000","last_fullsized":"2023-09-29T21:35:25.085897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3176,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:35:21.129625+0000","last_change":"2023-09-29T21:35:21.129625+0000","last_active":"2023-09-29T21:35:21.129625+0000","last_peered":"2023-09-29T21:35:21.129625+0000","last_clean":"2023-09-29T21:35:21.129625+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:35:21.129625+0000","last_undegraded":"2023-09-29T21:35:21.129625+0000","last_fullsized":"2023-09-29T21:35:21.129625+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740666,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:34:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.214,"15min":0.217},"min":{"1min":0.161,"5min":0.154,"15min":0.154},"max":{"1min":0.36799999999999999,"5min":0.38500000000000001,"15min":0.48999999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.23899999999999999,"15min":0.23799999999999999},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17000000000000001},"max":{"1min":0.38700000000000001,"5min":0.40799999999999997,"15min":0.47599999999999998},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24099999999999999},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.74099999999999999,"5min":1.044,"15min":1.044},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.28499999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740666,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.20300000000000001},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.31900000000000001,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.379,"5min":0.45200000000000001,"15min":0.502},"last":0.247}]},{"osd":2,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.184,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.193,"5min":0.193,"15min":0.17799999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.21199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740664,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.20699999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39400000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.16300000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35799999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.193}]},{"osd":2,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.221,"15min":0.223},"min":{"1min":0.188,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.371,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.27300000000000002,"15min":0.27400000000000002},"min":{"1min":0.23100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40699999999999997,"5min":0.47299999999999998,"15min":0.497},"last":0.312}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:35:55.539 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.539 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.539 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.539 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.540 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.540 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.540 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:35:55.540 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:35:55.840 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:55.840+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:56.108 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:56.108+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:56.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:56.860+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:57.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:57.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:57.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:57.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:58.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:58.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:58.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:58.842+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:59.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:35:59.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:35:59.809 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:35:59.809+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:00.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:00.140+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:00.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:00.775+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:01.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:01.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:01.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:01.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:02.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:02.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:02.749 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:02.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:03.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:03.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:03.779 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:03.779+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:04.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:04.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:04.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:04.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:05.156 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:05.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:05.734 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:05.734+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:06.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:06.160+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:06.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:06.709+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:07.134 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:07.134+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:07.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:07.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:08.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:08.157+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:08.672 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:08.672+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:09.141 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:09.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:09.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:09.680+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:10.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:10.138+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:10.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:10.721+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:11.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:11.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:11.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:11.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:12.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:12.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:12.720 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:12.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:13.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:13.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:13.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:13.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:14.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:14.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:14.710 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:14.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:15.196 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:15.196+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:15.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:36:15.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:15.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:15.855 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:36:15.857 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:36:15.869 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5896,"stamp":"2023-09-29T21:36:15.269950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001702"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":2998,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:02.152973+0000","last_change":"2023-09-29T21:36:02.152973+0000","last_active":"2023-09-29T21:36:02.152973+0000","last_peered":"2023-09-29T21:36:02.152973+0000","last_clean":"2023-09-29T21:36:02.152973+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:36:02.152973+0000","last_undegraded":"2023-09-29T21:36:02.152973+0000","last_fullsized":"2023-09-29T21:36:02.152973+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12748,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:07.134876+0000","last_change":"2023-09-29T21:36:07.134876+0000","last_active":"2023-09-29T21:36:07.134876+0000","last_peered":"2023-09-29T21:36:07.134876+0000","last_clean":"2023-09-29T21:36:07.134876+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:36:07.134876+0000","last_undegraded":"2023-09-29T21:36:07.134876+0000","last_fullsized":"2023-09-29T21:36:07.134876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":302,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:10.138203+0000","last_change":"2023-09-29T21:36:10.138203+0000","last_active":"2023-09-29T21:36:10.138203+0000","last_peered":"2023-09-29T21:36:10.138203+0000","last_clean":"2023-09-29T21:36:10.138203+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:36:10.138203+0000","last_undegraded":"2023-09-29T21:36:10.138203+0000","last_fullsized":"2023-09-29T21:36:10.138203+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3486,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:11.180342+0000","last_change":"2023-09-29T21:36:11.180342+0000","last_active":"2023-09-29T21:36:11.180342+0000","last_peered":"2023-09-29T21:36:11.180342+0000","last_clean":"2023-09-29T21:36:11.180342+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:36:11.180342+0000","last_undegraded":"2023-09-29T21:36:11.180342+0000","last_fullsized":"2023-09-29T21:36:11.180342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10824,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:05.156447+0000","last_change":"2023-09-29T21:36:05.156447+0000","last_active":"2023-09-29T21:36:05.156447+0000","last_peered":"2023-09-29T21:36:05.156447+0000","last_clean":"2023-09-29T21:36:05.156447+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:36:05.156447+0000","last_undegraded":"2023-09-29T21:36:05.156447+0000","last_fullsized":"2023-09-29T21:36:05.156447+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3182,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:12.215630+0000","last_change":"2023-09-29T21:36:12.215630+0000","last_active":"2023-09-29T21:36:12.215630+0000","last_peered":"2023-09-29T21:36:12.215630+0000","last_clean":"2023-09-29T21:36:12.215630+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:36:12.215630+0000","last_undegraded":"2023-09-29T21:36:12.215630+0000","last_fullsized":"2023-09-29T21:36:12.215630+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740670,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:00 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.222,"15min":0.217},"min":{"1min":0.16,"5min":0.154,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17399999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.45900000000000002},"last":0.22700000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.251,"15min":0.24099999999999999},"min":{"1min":0.19,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.435,"5min":1.044,"15min":1.044},"last":0.246},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.26300000000000001},"min":{"1min":0.218,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.27500000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740670,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.20300000000000001},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.31900000000000001,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.379,"5min":0.45200000000000001,"15min":0.502},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.184,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.193,"5min":0.193,"15min":0.17799999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.23300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740668,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.20699999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.114,"15min":0.114},"max":{"1min":0.39400000000000002,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.16300000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35799999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.24199999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:35:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.221,"15min":0.223},"min":{"1min":0.188,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.371,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.27300000000000002,"15min":0.27400000000000002},"min":{"1min":0.23100000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40699999999999997,"5min":0.47299999999999998,"15min":0.497},"last":0.32400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:36:15.871 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.871 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.871 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.872 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.872 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.872 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.872 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:15.872 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:36:16.243 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:16.243+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:16.709 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:16.710+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:17.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:17.218+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:17.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:17.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:18.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:18.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:18.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:18.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:19.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:19.231+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:19.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:19.811+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:20.230 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:20.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:20.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:20.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:21.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:21.260+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:21.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:21.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:22.251 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:22.252+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:22.854 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:22.855+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:23.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:23.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:23.885 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:23.885+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:24.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:24.278+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:24.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:24.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:25.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:25.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:25.915 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:25.916+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:26.336 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:26.336+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:26.877 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:26.877+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:27.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:27.332+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:27.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:27.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:28.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:28.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:28.863 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:28.863+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:29.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:29.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:29.847 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:29.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:30.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:30.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:30.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:30.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:31.359 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:31.359+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:31.838 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:31.838+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:32.313 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:32.313+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:32.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:32.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:33.330 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:33.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:33.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:33.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:34.366 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:34.366+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:34.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:34.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:35.367 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:35.368+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:35.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:36:35.925 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:35.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:36.191 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:36:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:36:36.204 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5906,"stamp":"2023-09-29T21:36:35.273046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3001,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:14.236162+0000","last_change":"2023-09-29T21:36:14.236162+0000","last_active":"2023-09-29T21:36:14.236162+0000","last_peered":"2023-09-29T21:36:14.236162+0000","last_clean":"2023-09-29T21:36:14.236162+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:36:14.236162+0000","last_undegraded":"2023-09-29T21:36:14.236162+0000","last_fullsized":"2023-09-29T21:36:14.236162+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12751,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:24.278881+0000","last_change":"2023-09-29T21:36:24.278881+0000","last_active":"2023-09-29T21:36:24.278881+0000","last_peered":"2023-09-29T21:36:24.278881+0000","last_clean":"2023-09-29T21:36:24.278881+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:36:24.278881+0000","last_undegraded":"2023-09-29T21:36:24.278881+0000","last_fullsized":"2023-09-29T21:36:24.278881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":305,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:28.376192+0000","last_change":"2023-09-29T21:36:28.376192+0000","last_active":"2023-09-29T21:36:28.376192+0000","last_peered":"2023-09-29T21:36:28.376192+0000","last_clean":"2023-09-29T21:36:28.376192+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:36:28.376192+0000","last_undegraded":"2023-09-29T21:36:28.376192+0000","last_fullsized":"2023-09-29T21:36:28.376192+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3489,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:31.359537+0000","last_change":"2023-09-29T21:36:31.359537+0000","last_active":"2023-09-29T21:36:31.359537+0000","last_peered":"2023-09-29T21:36:31.359537+0000","last_clean":"2023-09-29T21:36:31.359537+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:36:31.359537+0000","last_undegraded":"2023-09-29T21:36:31.359537+0000","last_fullsized":"2023-09-29T21:36:31.359537+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10827,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:18.206136+0000","last_change":"2023-09-29T21:36:18.206136+0000","last_active":"2023-09-29T21:36:18.206136+0000","last_peered":"2023-09-29T21:36:18.206136+0000","last_clean":"2023-09-29T21:36:18.206136+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:36:18.206136+0000","last_undegraded":"2023-09-29T21:36:18.206136+0000","last_fullsized":"2023-09-29T21:36:18.206136+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3182,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:12.215630+0000","last_change":"2023-09-29T21:36:12.215630+0000","last_active":"2023-09-29T21:36:12.215630+0000","last_peered":"2023-09-29T21:36:12.215630+0000","last_clean":"2023-09-29T21:36:12.215630+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:36:12.215630+0000","last_undegraded":"2023-09-29T21:36:12.215630+0000","last_fullsized":"2023-09-29T21:36:12.215630+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740674,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:00 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.222,"15min":0.217},"min":{"1min":0.16,"5min":0.154,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.185},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17399999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.45900000000000002},"last":0.22500000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.251,"15min":0.24099999999999999},"min":{"1min":0.19,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.435,"5min":1.044,"15min":1.044},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.26300000000000001},"min":{"1min":0.218,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.26500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.20300000000000001},"min":{"1min":0.152,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.31900000000000001,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.193},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.379,"5min":0.45200000000000001,"15min":0.502},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"min":{"1min":0.184,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33000000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.193,"5min":0.193,"15min":0.17799999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.307}]}]},{"osd":1,"up_from":8,"seq":34359740672,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20799999999999999,"15min":0.20599999999999999},"min":{"1min":0.14799999999999999,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.151},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.24399999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.39000000000000001,"5min":0.42699999999999999,"15min":0.443},"last":0.186}]},{"osd":2,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.16800000000000001,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.36699999999999999,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.23000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:36:36.206 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.206 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.206 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.207 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.207 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.207 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.207 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:36.207 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:36:36.323 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:36.323+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:36.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:36.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:37.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:37.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:37.878 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:37.878+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:38.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:38.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:38.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:38.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:39.335 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:39.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:39.899 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:39.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:40.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:40.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:40.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:40.921+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:41.322 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:41.322+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:41.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:41.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:42.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:42.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:42.862 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:42.862+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:43.418 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:43.419+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:43.866 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:43.867+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:44.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:44.463+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:44.818 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:44.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:45.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:45.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:45.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:45.798+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:46.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:46.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:46.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:46.792+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:47.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:47.476+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:47.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:47.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:48.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:48.468+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:48.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:48.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:49.476 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:49.475+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:49.761 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:49.761+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:50.454 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:50.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:50.791 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:50.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:51.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:51.424+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:51.829 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:51.828+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:52.380 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:52.379+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:52.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:52.780+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:53.372 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:53.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:53.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:53.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:54.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:54.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:54.747 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:54.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:55.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:55.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:55.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:55.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:56.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:36:56.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:56.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:56.571 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:36:56.573 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:36:56.585 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5916,"stamp":"2023-09-29T21:36:55.276325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001864"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3004,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:36.323619+0000","last_change":"2023-09-29T21:36:36.323619+0000","last_active":"2023-09-29T21:36:36.323619+0000","last_peered":"2023-09-29T21:36:36.323619+0000","last_clean":"2023-09-29T21:36:36.323619+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:36:36.323619+0000","last_undegraded":"2023-09-29T21:36:36.323619+0000","last_fullsized":"2023-09-29T21:36:36.323619+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12754,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:42.371811+0000","last_change":"2023-09-29T21:36:42.371811+0000","last_active":"2023-09-29T21:36:42.371811+0000","last_peered":"2023-09-29T21:36:42.371811+0000","last_clean":"2023-09-29T21:36:42.371811+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:36:42.371811+0000","last_undegraded":"2023-09-29T21:36:42.371811+0000","last_fullsized":"2023-09-29T21:36:42.371811+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":308,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:43.419333+0000","last_change":"2023-09-29T21:36:43.419333+0000","last_active":"2023-09-29T21:36:43.419333+0000","last_peered":"2023-09-29T21:36:43.419333+0000","last_clean":"2023-09-29T21:36:43.419333+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:36:43.419333+0000","last_undegraded":"2023-09-29T21:36:43.419333+0000","last_fullsized":"2023-09-29T21:36:43.419333+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3492,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:46.428779+0000","last_change":"2023-09-29T21:36:46.428779+0000","last_active":"2023-09-29T21:36:46.428779+0000","last_peered":"2023-09-29T21:36:46.428779+0000","last_clean":"2023-09-29T21:36:46.428779+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:36:46.428779+0000","last_undegraded":"2023-09-29T21:36:46.428779+0000","last_fullsized":"2023-09-29T21:36:46.428779+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10830,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:41.322857+0000","last_change":"2023-09-29T21:36:41.322857+0000","last_active":"2023-09-29T21:36:41.322857+0000","last_peered":"2023-09-29T21:36:41.322857+0000","last_clean":"2023-09-29T21:36:41.322857+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:36:41.322857+0000","last_undegraded":"2023-09-29T21:36:41.322857+0000","last_fullsized":"2023-09-29T21:36:41.322857+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12040,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521642+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521642+0000","last_peered":"2023-09-29T21:34:35.521642+0000","last_clean":"2023-09-29T21:34:35.521642+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521642+0000","last_undegraded":"2023-09-29T21:34:35.521642+0000","last_fullsized":"2023-09-29T21:34:35.521642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3188,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:36:48.469851+0000","last_change":"2023-09-29T21:36:48.469851+0000","last_active":"2023-09-29T21:36:48.469851+0000","last_peered":"2023-09-29T21:36:48.469851+0000","last_clean":"2023-09-29T21:36:48.469851+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:36:48.469851+0000","last_undegraded":"2023-09-29T21:36:48.469851+0000","last_fullsized":"2023-09-29T21:36:48.469851+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740678,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:00 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.222,"15min":0.217},"min":{"1min":0.16,"5min":0.154,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.218},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17399999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.45900000000000002},"last":0.23100000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.251,"15min":0.24099999999999999},"min":{"1min":0.19,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.435,"5min":1.044,"15min":1.044},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.26300000000000001},"min":{"1min":0.218,"5min":0.20599999999999999,"15min":0.17799999999999999},"max":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.27600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.19900000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.16500000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.27100000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.38600000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.254}]},{"osd":2,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33600000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.247,"5min":0.255,"15min":0.254},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36099999999999999,"5min":0.504,"15min":0.504},"last":0.223}]}]},{"osd":1,"up_from":8,"seq":34359740676,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20799999999999999,"15min":0.20599999999999999},"min":{"1min":0.14799999999999999,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.24399999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.39000000000000001,"5min":0.42699999999999999,"15min":0.443},"last":0.247}]},{"osd":2,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.16800000000000001,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.36699999999999999,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.192},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.26000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:36:56.587 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.587 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.587 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.588 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.588 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.588 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.588 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:36:56.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:36:56.687 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:56.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:56.922 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:36:56.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:36:57.249 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:36:57.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:36:57.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:57.339+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:36:57.640 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:36:57.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:57.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:57.977 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:36:57.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:36:58.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:58.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:58.394 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:36:58.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:36:58.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:58.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:58.720 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:36:58.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:36:59.055 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:36:59.066 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:36:59.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:36:59.281+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:36:59.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:36:59.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:00.297 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:00.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:00.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:00.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:01.347 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:01.346+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:01.685 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:01.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:02.346 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:02.345+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:02.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:02.681+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:03.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:03.302+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:03.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:03.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:04.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:04.270+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:04.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:04.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:05.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:05.275+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:05.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:05.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:06.255 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:06.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:06.635 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:06.634+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:07.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:07.221+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:07.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:07.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:08.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:08.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:08.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:08.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:09.281 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:09.280+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:09.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:09.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:10.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:10.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:10.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:10.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:11.250 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:11.250+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:11.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:11.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:12.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:12.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:12.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:12.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:13.326 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:13.325+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:13.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:13.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:14.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:14.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:14.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:14.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:15.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:15.277+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:15.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:15.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:16.275 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:16.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:16.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:16.673+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:17.247 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:17.247+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:17.721 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:17.720+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:18.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:18.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:18.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:18.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:19.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:37:19.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:19.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:37:19.390 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:37:19.400 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5928,"stamp":"2023-09-29T21:37:19.280469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002163"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3011,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:08.268553+0000","last_change":"2023-09-29T21:37:08.268553+0000","last_active":"2023-09-29T21:37:08.268553+0000","last_peered":"2023-09-29T21:37:08.268553+0000","last_clean":"2023-09-29T21:37:08.268553+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:37:08.268553+0000","last_undegraded":"2023-09-29T21:37:08.268553+0000","last_fullsized":"2023-09-29T21:37:08.268553+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12761,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:10.268835+0000","last_change":"2023-09-29T21:37:10.268835+0000","last_active":"2023-09-29T21:37:10.268835+0000","last_peered":"2023-09-29T21:37:10.268835+0000","last_clean":"2023-09-29T21:37:10.268835+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:37:10.268835+0000","last_undegraded":"2023-09-29T21:37:10.268835+0000","last_fullsized":"2023-09-29T21:37:10.268835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:13.326306+0000","last_change":"2023-09-29T21:37:13.326306+0000","last_active":"2023-09-29T21:37:13.326306+0000","last_peered":"2023-09-29T21:37:13.326306+0000","last_clean":"2023-09-29T21:37:13.326306+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:37:13.326306+0000","last_undegraded":"2023-09-29T21:37:13.326306+0000","last_fullsized":"2023-09-29T21:37:13.326306+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3499,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:14.322311+0000","last_change":"2023-09-29T21:37:14.322311+0000","last_active":"2023-09-29T21:37:14.322311+0000","last_peered":"2023-09-29T21:37:14.322311+0000","last_clean":"2023-09-29T21:37:14.322311+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:37:14.322311+0000","last_undegraded":"2023-09-29T21:37:14.322311+0000","last_fullsized":"2023-09-29T21:37:14.322311+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10837,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:09.281610+0000","last_change":"2023-09-29T21:37:09.281610+0000","last_active":"2023-09-29T21:37:09.281610+0000","last_peered":"2023-09-29T21:37:09.281610+0000","last_clean":"2023-09-29T21:37:09.281610+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:37:09.281610+0000","last_undegraded":"2023-09-29T21:37:09.281610+0000","last_fullsized":"2023-09-29T21:37:09.281610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3192,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:06.256104+0000","last_change":"2023-09-29T21:37:06.256104+0000","last_active":"2023-09-29T21:37:06.256104+0000","last_peered":"2023-09-29T21:37:06.256104+0000","last_clean":"2023-09-29T21:37:06.256104+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:37:06.256104+0000","last_undegraded":"2023-09-29T21:37:06.256104+0000","last_fullsized":"2023-09-29T21:37:06.256104+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740683,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22500000000000001,"15min":0.218},"min":{"1min":0.17299999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.36599999999999999,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.247,"15min":0.23899999999999999},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.38,"5min":0.442,"15min":0.45900000000000002},"last":0.247}]},{"osd":1,"last update":"Fri Sep 29 21:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.255,"15min":0.24099999999999999},"min":{"1min":0.185,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":1.044,"15min":1.044},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.443,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.27300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.19900000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.184},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.27100000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.38600000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.249}]},{"osd":2,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33600000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.247,"5min":0.255,"15min":0.254},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36099999999999999,"5min":0.504,"15min":0.504},"last":0.214}]}]},{"osd":1,"up_from":8,"seq":34359740681,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20799999999999999,"15min":0.20599999999999999},"min":{"1min":0.14799999999999999,"5min":0.114,"15min":0.114},"max":{"1min":0.35699999999999998,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.24399999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.39000000000000001,"5min":0.42699999999999999,"15min":0.443},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.16800000000000001,"5min":0.16400000000000001,"15min":0.158},"max":{"1min":0.36699999999999999,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.216},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.216,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.497},"last":0.26000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:37:19.402 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.402 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.403 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.403 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.403 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.403 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.403 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:19.404 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:37:19.738 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:19.737+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:20.320 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:20.319+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:20.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:20.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:21.344 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:21.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:21.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:21.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:22.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:22.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:22.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:22.759+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:23.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:23.273+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:23.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:23.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:24.298 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:24.297+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:24.728 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:24.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:25.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:25.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:25.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:25.688+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:26.217 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:26.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:26.659 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:26.658+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:27.220 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:27.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:27.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:27.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:28.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:28.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:28.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:28.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:29.228 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:29.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:29.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:29.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:30.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:30.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:30.576 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:30.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:31.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:31.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:31.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:31.618+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:32.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:32.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:32.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:32.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:33.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:33.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:33.646 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:33.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:34.186 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:34.185+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:34.680 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:34.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:35.151 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:35.150+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:35.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:35.631+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:36.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:36.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:36.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:36.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:37.132 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:37.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:37.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:37.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:38.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:38.105+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:38.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:38.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:39.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:39.081+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:39.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:37:39.602 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:39.601+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:39.726 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:37:39.728 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:37:39.740 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5938,"stamp":"2023-09-29T21:37:39.283746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002028"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3014,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:20.320848+0000","last_change":"2023-09-29T21:37:20.320848+0000","last_active":"2023-09-29T21:37:20.320848+0000","last_peered":"2023-09-29T21:37:20.320848+0000","last_clean":"2023-09-29T21:37:20.320848+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:37:20.320848+0000","last_undegraded":"2023-09-29T21:37:20.320848+0000","last_fullsized":"2023-09-29T21:37:20.320848+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12764,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:30.268205+0000","last_change":"2023-09-29T21:37:30.268205+0000","last_active":"2023-09-29T21:37:30.268205+0000","last_peered":"2023-09-29T21:37:30.268205+0000","last_clean":"2023-09-29T21:37:30.268205+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:37:30.268205+0000","last_undegraded":"2023-09-29T21:37:30.268205+0000","last_fullsized":"2023-09-29T21:37:30.268205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:34.186920+0000","last_change":"2023-09-29T21:37:34.186920+0000","last_active":"2023-09-29T21:37:34.186920+0000","last_peered":"2023-09-29T21:37:34.186920+0000","last_clean":"2023-09-29T21:37:34.186920+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:37:34.186920+0000","last_undegraded":"2023-09-29T21:37:34.186920+0000","last_fullsized":"2023-09-29T21:37:34.186920+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3502,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:35.151944+0000","last_change":"2023-09-29T21:37:35.151944+0000","last_active":"2023-09-29T21:37:35.151944+0000","last_peered":"2023-09-29T21:37:35.151944+0000","last_clean":"2023-09-29T21:37:35.151944+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:37:35.151944+0000","last_undegraded":"2023-09-29T21:37:35.151944+0000","last_fullsized":"2023-09-29T21:37:35.151944+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10840,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:22.321775+0000","last_change":"2023-09-29T21:37:22.321775+0000","last_active":"2023-09-29T21:37:22.321775+0000","last_peered":"2023-09-29T21:37:22.321775+0000","last_clean":"2023-09-29T21:37:22.321775+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:37:22.321775+0000","last_undegraded":"2023-09-29T21:37:22.321775+0000","last_fullsized":"2023-09-29T21:37:22.321775+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3195,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:18.286253+0000","last_change":"2023-09-29T21:37:18.286253+0000","last_active":"2023-09-29T21:37:18.286253+0000","last_peered":"2023-09-29T21:37:18.286253+0000","last_clean":"2023-09-29T21:37:18.286253+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:37:18.286253+0000","last_undegraded":"2023-09-29T21:37:18.286253+0000","last_fullsized":"2023-09-29T21:37:18.286253+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740687,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22500000000000001,"15min":0.218},"min":{"1min":0.17299999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.36599999999999999,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.247,"15min":0.23899999999999999},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.38,"5min":0.442,"15min":0.45900000000000002},"last":0.312}]},{"osd":1,"last update":"Fri Sep 29 21:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.255,"15min":0.24099999999999999},"min":{"1min":0.185,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":1.044,"15min":1.044},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.443,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.33700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.19900000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.42499999999999999},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.27100000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.38600000000000001,"5min":0.45200000000000001,"15min":0.502},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17000000000000001},"max":{"1min":0.33600000000000002,"5min":0.40699999999999997,"15min":0.42299999999999999},"last":0.248},{"interface":"front","average":{"1min":0.247,"5min":0.255,"15min":0.254},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36099999999999999,"5min":0.504,"15min":0.504},"last":0.26500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740685,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.151,"5min":0.13700000000000001,"15min":0.114},"max":{"1min":0.32400000000000001,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.245,"15min":0.23899999999999999},"min":{"1min":0.186,"5min":0.16200000000000001,"15min":0.13600000000000001},"max":{"1min":0.35699999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.23599999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.155,"5min":0.155,"15min":0.155},"max":{"1min":0.313,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.34999999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:37:39.741 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.742 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.742 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.742 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.742 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.742 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.743 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:37:39.743 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:37:40.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:40.038+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:40.623 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:40.622+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:41.032 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:41.031+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:41.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:41.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:41.995 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:41.994+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:42.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:42.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:42.990 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:42.989+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:43.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:43.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:43.968 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:43.967+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:44.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:44.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:45.018 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:45.017+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:45.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:45.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:46.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:46.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:46.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:46.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:47.054 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:47.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:47.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:47.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:48.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:48.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:48.688 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:48.687+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:49.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:49.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:49.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:49.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:50.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:50.047+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:50.668 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:50.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:51.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:51.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:51.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:51.643+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:52.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:52.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:52.678 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:52.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:53.042 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:53.041+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:53.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:53.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:54.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:54.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:54.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:54.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:55.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:55.050+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:55.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:55.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:56.006 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:56.006+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:56.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:56.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:57.052 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:57.052+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:57.706 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:57.706+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:58.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:58.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:58.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:58.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:59.035 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:37:59.036+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:59.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:37:59.742+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:37:59.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:38:00.070 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:38:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:38:00.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:00.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:00.084 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5948,"stamp":"2023-09-29T21:37:59.286491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001583"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3017,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:43.968760+0000","last_change":"2023-09-29T21:37:43.968760+0000","last_active":"2023-09-29T21:37:43.968760+0000","last_peered":"2023-09-29T21:37:43.968760+0000","last_clean":"2023-09-29T21:37:43.968760+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:37:43.968760+0000","last_undegraded":"2023-09-29T21:37:43.968760+0000","last_fullsized":"2023-09-29T21:37:43.968760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12767,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:52.061170+0000","last_change":"2023-09-29T21:37:52.061170+0000","last_active":"2023-09-29T21:37:52.061170+0000","last_peered":"2023-09-29T21:37:52.061170+0000","last_clean":"2023-09-29T21:37:52.061170+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:37:52.061170+0000","last_undegraded":"2023-09-29T21:37:52.061170+0000","last_fullsized":"2023-09-29T21:37:52.061170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":321,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:54.082619+0000","last_change":"2023-09-29T21:37:54.082619+0000","last_active":"2023-09-29T21:37:54.082619+0000","last_peered":"2023-09-29T21:37:54.082619+0000","last_clean":"2023-09-29T21:37:54.082619+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:37:54.082619+0000","last_undegraded":"2023-09-29T21:37:54.082619+0000","last_fullsized":"2023-09-29T21:37:54.082619+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3505,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:55.050402+0000","last_change":"2023-09-29T21:37:55.050402+0000","last_active":"2023-09-29T21:37:55.050402+0000","last_peered":"2023-09-29T21:37:55.050402+0000","last_clean":"2023-09-29T21:37:55.050402+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:37:55.050402+0000","last_undegraded":"2023-09-29T21:37:55.050402+0000","last_fullsized":"2023-09-29T21:37:55.050402+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10843,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:46.054651+0000","last_change":"2023-09-29T21:37:46.054651+0000","last_active":"2023-09-29T21:37:46.054651+0000","last_peered":"2023-09-29T21:37:46.054651+0000","last_clean":"2023-09-29T21:37:46.054651+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:37:46.054651+0000","last_undegraded":"2023-09-29T21:37:46.054651+0000","last_fullsized":"2023-09-29T21:37:46.054651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3201,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:37:57.052488+0000","last_change":"2023-09-29T21:37:57.052488+0000","last_active":"2023-09-29T21:37:57.052488+0000","last_peered":"2023-09-29T21:37:57.052488+0000","last_clean":"2023-09-29T21:37:57.052488+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:37:57.052488+0000","last_undegraded":"2023-09-29T21:37:57.052488+0000","last_fullsized":"2023-09-29T21:37:57.052488+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740691,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.224,"5min":0.22500000000000001,"15min":0.218},"min":{"1min":0.17299999999999999,"5min":0.154,"15min":0.154},"max":{"1min":0.36599999999999999,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.185},{"interface":"front","average":{"1min":0.245,"5min":0.247,"15min":0.23899999999999999},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.38,"5min":0.442,"15min":0.45900000000000002},"last":0.22}]},{"osd":1,"last update":"Fri Sep 29 21:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.255,"15min":0.24099999999999999},"min":{"1min":0.185,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":1.044,"15min":1.044},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.443,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.252}]}]},{"osd":0,"up_from":8,"seq":34359740691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.20200000000000001,"15min":0.20000000000000001},"min":{"1min":0.16,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.42099999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.26900000000000002},"min":{"1min":0.21299999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.317,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.251},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35299999999999998,"5min":0.504,"15min":0.504},"last":0.48899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740689,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.151,"5min":0.13700000000000001,"15min":0.114},"max":{"1min":0.32400000000000001,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.245,"15min":0.23899999999999999},"min":{"1min":0.186,"5min":0.16200000000000001,"15min":0.13600000000000001},"max":{"1min":0.35699999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.245}]},{"osd":2,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.155,"5min":0.155,"15min":0.155},"max":{"1min":0.313,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.219},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.34999999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.27800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:38:00.085 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.086 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.086 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.086 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.086 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.087 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.087 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:00.087 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:38:00.760 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:00.760+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:01.060 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:01.060+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:01.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:01.749+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:02.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:02.073+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:02.713 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:02.713+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:03.104 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:03.104+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:03.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:03.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:04.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:04.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:04.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:04.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:05.028 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:05.028+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:05.657 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:05.657+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:06.002 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:06.002+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:06.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:06.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:07.003 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:07.003+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:07.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:07.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:07.978 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:07.978+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:08.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:08.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:09.017 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:09.018+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:09.692 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:09.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:10.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:10.040+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:10.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:10.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:11.053 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:11.053+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:11.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:11.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:12.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:12.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:12.624 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:12.624+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:13.048 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:13.048+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:13.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:13.670+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:14.067 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:14.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:14.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:14.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:15.093 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:15.093+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:15.627 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:15.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:16.119 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:16.119+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:16.652 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:16.652+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:17.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:17.157+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:17.662 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:17.662+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:18.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:18.142+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:18.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:18.679+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:19.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:19.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:19.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:19.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:20.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:38:20.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:20.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:20.429 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:38:20.431 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:38:20.443 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5958,"stamp":"2023-09-29T21:38:19.289543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001764"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3023,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:15.093969+0000","last_change":"2023-09-29T21:38:15.093969+0000","last_active":"2023-09-29T21:38:15.093969+0000","last_peered":"2023-09-29T21:38:15.093969+0000","last_clean":"2023-09-29T21:38:15.093969+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:38:15.093969+0000","last_undegraded":"2023-09-29T21:38:15.093969+0000","last_fullsized":"2023-09-29T21:38:15.093969+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12770,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:04.068555+0000","last_change":"2023-09-29T21:38:04.068555+0000","last_active":"2023-09-29T21:38:04.068555+0000","last_peered":"2023-09-29T21:38:04.068555+0000","last_clean":"2023-09-29T21:38:04.068555+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:38:04.068555+0000","last_undegraded":"2023-09-29T21:38:04.068555+0000","last_fullsized":"2023-09-29T21:38:04.068555+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":324,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:06.002968+0000","last_change":"2023-09-29T21:38:06.002968+0000","last_active":"2023-09-29T21:38:06.002968+0000","last_peered":"2023-09-29T21:38:06.002968+0000","last_clean":"2023-09-29T21:38:06.002968+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:38:06.002968+0000","last_undegraded":"2023-09-29T21:38:06.002968+0000","last_fullsized":"2023-09-29T21:38:06.002968+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3508,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:07.004088+0000","last_change":"2023-09-29T21:38:07.004088+0000","last_active":"2023-09-29T21:38:07.004088+0000","last_peered":"2023-09-29T21:38:07.004088+0000","last_clean":"2023-09-29T21:38:07.004088+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:38:07.004088+0000","last_undegraded":"2023-09-29T21:38:07.004088+0000","last_fullsized":"2023-09-29T21:38:07.004088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10849,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:17.157650+0000","last_change":"2023-09-29T21:38:17.157650+0000","last_active":"2023-09-29T21:38:17.157650+0000","last_peered":"2023-09-29T21:38:17.157650+0000","last_clean":"2023-09-29T21:38:17.157650+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:38:17.157650+0000","last_undegraded":"2023-09-29T21:38:17.157650+0000","last_fullsized":"2023-09-29T21:38:17.157650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3204,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:13.048820+0000","last_change":"2023-09-29T21:38:13.048820+0000","last_active":"2023-09-29T21:38:13.048820+0000","last_peered":"2023-09-29T21:38:13.048820+0000","last_clean":"2023-09-29T21:38:13.048820+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:38:13.048820+0000","last_undegraded":"2023-09-29T21:38:13.048820+0000","last_fullsized":"2023-09-29T21:38:13.048820+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740695,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.219},"min":{"1min":0.16500000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.32200000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.255,"5min":0.251,"15min":0.23999999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34399999999999997,"5min":0.442,"15min":0.45900000000000002},"last":0.19600000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.253,"15min":0.24199999999999999},"min":{"1min":0.187,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.32600000000000001,"5min":0.74099999999999999,"15min":1.044},"last":0.217},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.26400000000000001},"min":{"1min":0.20799999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.376,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.23100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.20200000000000001,"15min":0.20000000000000001},"min":{"1min":0.16,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.42099999999999999},"last":0.159},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.26900000000000002},"min":{"1min":0.21299999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.214}]},{"osd":2,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.317,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.251},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35299999999999998,"5min":0.504,"15min":0.504},"last":0.24099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740693,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.21099999999999999,"15min":0.20699999999999999},"min":{"1min":0.151,"5min":0.13700000000000001,"15min":0.114},"max":{"1min":0.32400000000000001,"5min":0.39700000000000002,"15min":0.40999999999999998},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.245,"15min":0.23899999999999999},"min":{"1min":0.186,"5min":0.16200000000000001,"15min":0.13600000000000001},"max":{"1min":0.35699999999999998,"5min":0.42699999999999999,"15min":0.443},"last":0.26000000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.155,"5min":0.155,"15min":0.155},"max":{"1min":0.313,"5min":0.40799999999999997,"15min":0.44700000000000001},"last":0.215},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.34999999999999998,"5min":0.47299999999999998,"15min":0.497},"last":0.26800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:38:20.445 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.445 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.445 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.445 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.446 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.446 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.446 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:20.446 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:38:20.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:20.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:21.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:21.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:21.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:21.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:22.268 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:22.268+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:22.641 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:22.641+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:23.249 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:23.249+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:23.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:23.611+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:24.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:24.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:24.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:24.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:25.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:25.311+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:25.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:25.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:26.340 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:26.340+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:26.663 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:26.664+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:27.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:27.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:27.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:27.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:28.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:28.382+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:28.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:28.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:29.423 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:29.423+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:29.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:29.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:30.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:30.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:30.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:30.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:31.352 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:31.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:31.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:31.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:32.364 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:32.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:32.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:32.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:33.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:33.328+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:33.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:33.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:34.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:34.309+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:34.493 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:34.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:35.342 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:35.342+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:35.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:35.482+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:36.308 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:36.308+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:36.520 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:36.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:37.266 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:37.266+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:37.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:37.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:38.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:38.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:38.481 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:38.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:39.350 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:39.350+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:39.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:39.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:40.396 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:40.396+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:40.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:38:40.479 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:40.480+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:40.760 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:38:40.762 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:38:40.773 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5968,"stamp":"2023-09-29T21:38:39.292351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001709"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3026,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:30.393834+0000","last_change":"2023-09-29T21:38:30.393834+0000","last_active":"2023-09-29T21:38:30.393834+0000","last_peered":"2023-09-29T21:38:30.393834+0000","last_clean":"2023-09-29T21:38:30.393834+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:38:30.393834+0000","last_undegraded":"2023-09-29T21:38:30.393834+0000","last_fullsized":"2023-09-29T21:38:30.393834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12776,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:33.328802+0000","last_change":"2023-09-29T21:38:33.328802+0000","last_active":"2023-09-29T21:38:33.328802+0000","last_peered":"2023-09-29T21:38:33.328802+0000","last_clean":"2023-09-29T21:38:33.328802+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:38:33.328802+0000","last_undegraded":"2023-09-29T21:38:33.328802+0000","last_fullsized":"2023-09-29T21:38:33.328802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":330,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:35.342266+0000","last_change":"2023-09-29T21:38:35.342266+0000","last_active":"2023-09-29T21:38:35.342266+0000","last_peered":"2023-09-29T21:38:35.342266+0000","last_clean":"2023-09-29T21:38:35.342266+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:38:35.342266+0000","last_undegraded":"2023-09-29T21:38:35.342266+0000","last_fullsized":"2023-09-29T21:38:35.342266+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3514,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:36.308693+0000","last_change":"2023-09-29T21:38:36.308693+0000","last_active":"2023-09-29T21:38:36.308693+0000","last_peered":"2023-09-29T21:38:36.308693+0000","last_clean":"2023-09-29T21:38:36.308693+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:38:36.308693+0000","last_undegraded":"2023-09-29T21:38:36.308693+0000","last_fullsized":"2023-09-29T21:38:36.308693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10852,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:31.352831+0000","last_change":"2023-09-29T21:38:31.352831+0000","last_active":"2023-09-29T21:38:31.352831+0000","last_peered":"2023-09-29T21:38:31.352831+0000","last_clean":"2023-09-29T21:38:31.352831+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:38:31.352831+0000","last_undegraded":"2023-09-29T21:38:31.352831+0000","last_fullsized":"2023-09-29T21:38:31.352831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3207,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:29.423843+0000","last_change":"2023-09-29T21:38:29.423843+0000","last_active":"2023-09-29T21:38:29.423843+0000","last_peered":"2023-09-29T21:38:29.423843+0000","last_clean":"2023-09-29T21:38:29.423843+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:38:29.423843+0000","last_undegraded":"2023-09-29T21:38:29.423843+0000","last_fullsized":"2023-09-29T21:38:29.423843+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740699,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.219},"min":{"1min":0.16500000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.32200000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.255,"5min":0.251,"15min":0.23999999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34399999999999997,"5min":0.442,"15min":0.45900000000000002},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Sep 29 21:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.253,"15min":0.24199999999999999},"min":{"1min":0.187,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.32600000000000001,"5min":0.74099999999999999,"15min":1.044},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.26400000000000001},"min":{"1min":0.20799999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.376,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.36399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.20200000000000001,"15min":0.20000000000000001},"min":{"1min":0.16,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.42099999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.26900000000000002},"min":{"1min":0.21299999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.31900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.317,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.246},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.251},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35299999999999998,"5min":0.504,"15min":0.504},"last":0.27500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740697,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.17000000000000001,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.188},{"interface":"front","average":{"1min":0.245,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.223},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.497},"last":0.22800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:38:40.775 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.775 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.775 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.775 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.775 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.776 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.776 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:38:40.776 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:38:41.394 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:41.394+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:41.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:41.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:42.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:42.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:42.483 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:42.483+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:43.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:43.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:43.464 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:43.465+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:44.337 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:44.338+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:44.439 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:44.440+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:45.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:45.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:45.444 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:45.444+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:46.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:46.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:46.470 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:46.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:47.271 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:47.271+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:47.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:47.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:48.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:48.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:48.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:48.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:49.306 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:49.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:49.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:49.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:50.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:50.347+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:50.585 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:50.584+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:51.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:51.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:51.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:51.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:52.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:52.289+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:52.583 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:52.582+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:53.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:53.277+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:53.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:53.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:54.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:54.230+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:54.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:54.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:55.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:55.217+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:55.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:55.539+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:56.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:56.177+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:56.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:56.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:57.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:57.143+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:57.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:57.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:58.096 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:58.095+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:58.535 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:58.535+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:59.069 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:38:59.069+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:38:59.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:38:59.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:00.118 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:00.118+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:00.448 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:00.447+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:00.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:39:01.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:39:01.091 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:39:01.102 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5978,"stamp":"2023-09-29T21:38:59.295258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001740"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3029,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:50.348496+0000","last_change":"2023-09-29T21:38:50.348496+0000","last_active":"2023-09-29T21:38:50.348496+0000","last_peered":"2023-09-29T21:38:50.348496+0000","last_clean":"2023-09-29T21:38:50.348496+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:38:50.348496+0000","last_undegraded":"2023-09-29T21:38:50.348496+0000","last_fullsized":"2023-09-29T21:38:50.348496+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12779,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:53.278541+0000","last_change":"2023-09-29T21:38:53.278541+0000","last_active":"2023-09-29T21:38:53.278541+0000","last_peered":"2023-09-29T21:38:53.278541+0000","last_clean":"2023-09-29T21:38:53.278541+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:38:53.278541+0000","last_undegraded":"2023-09-29T21:38:53.278541+0000","last_fullsized":"2023-09-29T21:38:53.278541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":333,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:54.231791+0000","last_change":"2023-09-29T21:38:54.231791+0000","last_active":"2023-09-29T21:38:54.231791+0000","last_peered":"2023-09-29T21:38:54.231791+0000","last_clean":"2023-09-29T21:38:54.231791+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:38:54.231791+0000","last_undegraded":"2023-09-29T21:38:54.231791+0000","last_fullsized":"2023-09-29T21:38:54.231791+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3517,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:56.178247+0000","last_change":"2023-09-29T21:38:56.178247+0000","last_active":"2023-09-29T21:38:56.178247+0000","last_peered":"2023-09-29T21:38:56.178247+0000","last_clean":"2023-09-29T21:38:56.178247+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:38:56.178247+0000","last_undegraded":"2023-09-29T21:38:56.178247+0000","last_fullsized":"2023-09-29T21:38:56.178247+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10855,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:52.291013+0000","last_change":"2023-09-29T21:38:52.291013+0000","last_active":"2023-09-29T21:38:52.291013+0000","last_peered":"2023-09-29T21:38:52.291013+0000","last_clean":"2023-09-29T21:38:52.291013+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:38:52.291013+0000","last_undegraded":"2023-09-29T21:38:52.291013+0000","last_fullsized":"2023-09-29T21:38:52.291013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12041,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:34:35.521713+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:34:35.521713+0000","last_peered":"2023-09-29T21:34:35.521713+0000","last_clean":"2023-09-29T21:34:35.521713+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:34:35.521713+0000","last_undegraded":"2023-09-29T21:34:35.521713+0000","last_fullsized":"2023-09-29T21:34:35.521713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 143s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3213,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:38:57.144556+0000","last_change":"2023-09-29T21:38:57.144556+0000","last_active":"2023-09-29T21:38:57.144556+0000","last_peered":"2023-09-29T21:38:57.144556+0000","last_clean":"2023-09-29T21:38:57.144556+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:38:57.144556+0000","last_undegraded":"2023-09-29T21:38:57.144556+0000","last_fullsized":"2023-09-29T21:38:57.144556+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740703,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.219},"min":{"1min":0.16500000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.32200000000000001,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.223},{"interface":"front","average":{"1min":0.255,"5min":0.251,"15min":0.23999999999999999},"min":{"1min":0.188,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34399999999999997,"5min":0.442,"15min":0.45900000000000002},"last":0.27600000000000002}]},{"osd":1,"last update":"Fri Sep 29 21:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.253,"15min":0.24199999999999999},"min":{"1min":0.187,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.32600000000000001,"5min":0.74099999999999999,"15min":1.044},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.26400000000000001},"min":{"1min":0.20799999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.376,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.23699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.20300000000000001,"15min":0.19900000000000001},"min":{"1min":0.153,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.42099999999999999},"last":0.16800000000000001},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27000000000000002},"min":{"1min":0.21099999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.183,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.254},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.20499999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48899999999999999,"5min":0.504,"15min":0.504},"last":0.224}]}]},{"osd":1,"up_from":8,"seq":34359740701,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.17000000000000001,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.214},{"interface":"front","average":{"1min":0.245,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.248}]},{"osd":2,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.223},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.497},"last":0.27100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:39:01.103 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.104 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.104 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.104 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.104 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.104 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.105 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:01.105 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:39:01.105 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:01.103+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:01.424 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:39:01.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:39:01.458 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:01.457+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:01.762 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:39:01.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:39:02.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:02.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:39:02.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:39:02.471 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:02.470+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:02.483 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:39:02.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:39:02.813 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:39:02.824 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:39:03.082 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:03.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:03.182 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:39:03.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:39:03.429 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:03.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:39:03.554 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:39:04.102 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:04.101+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:04.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:04.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:05.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:05.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:05.413 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:05.412+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:06.116 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:06.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:06.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:06.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:07.159 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:07.158+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:07.357 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:07.356+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:08.150 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:08.149+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:08.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:08.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:09.139 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:09.139+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:09.264 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:09.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:10.115 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:10.114+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:10.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:10.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:11.080 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:11.079+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:11.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:11.212+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:12.083 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:12.082+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:12.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:12.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:13.078 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:13.077+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:13.218 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:13.218+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:14.085 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:14.085+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:14.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:14.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:15.126 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:15.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:15.228 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:15.227+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:16.138 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:16.136+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:16.231 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:16.231+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:17.125 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:17.124+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:17.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:17.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:18.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:18.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:18.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:18.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:19.113 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:19.112+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:19.236 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:19.235+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:20.112 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:20.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:20.237 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:20.236+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:21.160 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:21.159+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:21.229 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:21.228+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:22.170 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:22.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:22.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:22.267+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:23.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:23.180+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:23.276 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:23.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:23.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:39:23.873 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:39:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:39:23.886 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":5990,"stamp":"2023-09-29T21:39:23.298759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001732"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3033,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:05.107327+0000","last_change":"2023-09-29T21:39:05.107327+0000","last_active":"2023-09-29T21:39:05.107327+0000","last_peered":"2023-09-29T21:39:05.107327+0000","last_clean":"2023-09-29T21:39:05.107327+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:39:05.107327+0000","last_undegraded":"2023-09-29T21:39:05.107327+0000","last_fullsized":"2023-09-29T21:39:05.107327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12783,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:14.085929+0000","last_change":"2023-09-29T21:39:14.085929+0000","last_active":"2023-09-29T21:39:14.085929+0000","last_peered":"2023-09-29T21:39:14.085929+0000","last_clean":"2023-09-29T21:39:14.085929+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:39:14.085929+0000","last_undegraded":"2023-09-29T21:39:14.085929+0000","last_fullsized":"2023-09-29T21:39:14.085929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":337,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:16.138194+0000","last_change":"2023-09-29T21:39:16.138194+0000","last_active":"2023-09-29T21:39:16.138194+0000","last_peered":"2023-09-29T21:39:16.138194+0000","last_clean":"2023-09-29T21:39:16.138194+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:39:16.138194+0000","last_undegraded":"2023-09-29T21:39:16.138194+0000","last_fullsized":"2023-09-29T21:39:16.138194+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3521,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:18.142177+0000","last_change":"2023-09-29T21:39:18.142177+0000","last_active":"2023-09-29T21:39:18.142177+0000","last_peered":"2023-09-29T21:39:18.142177+0000","last_clean":"2023-09-29T21:39:18.142177+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:39:18.142177+0000","last_undegraded":"2023-09-29T21:39:18.142177+0000","last_fullsized":"2023-09-29T21:39:18.142177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10859,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:08.150836+0000","last_change":"2023-09-29T21:39:08.150836+0000","last_active":"2023-09-29T21:39:08.150836+0000","last_peered":"2023-09-29T21:39:08.150836+0000","last_clean":"2023-09-29T21:39:08.150836+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:39:08.150836+0000","last_undegraded":"2023-09-29T21:39:08.150836+0000","last_fullsized":"2023-09-29T21:39:08.150836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3217,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:21.160965+0000","last_change":"2023-09-29T21:39:21.160965+0000","last_active":"2023-09-29T21:39:21.160965+0000","last_peered":"2023-09-29T21:39:21.160965+0000","last_clean":"2023-09-29T21:39:21.160965+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:39:21.160965+0000","last_undegraded":"2023-09-29T21:39:21.160965+0000","last_fullsized":"2023-09-29T21:39:21.160965+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740708,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.218},"min":{"1min":0.17000000000000001,"5min":0.16,"15min":0.154},"max":{"1min":0.33700000000000002,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.216},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25,"15min":0.23999999999999999},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40500000000000003,"5min":0.442,"15min":0.45900000000000002},"last":0.23000000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.253,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.74099999999999999,"15min":1.044},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27700000000000002,"15min":0.26400000000000001},"min":{"1min":0.21099999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.39600000000000002,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.26200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.20300000000000001,"15min":0.19900000000000001},"min":{"1min":0.153,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.42099999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27000000000000002},"min":{"1min":0.21099999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.224}]},{"osd":2,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.183,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.161},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.20499999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48899999999999999,"5min":0.504,"15min":0.504},"last":0.20599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740706,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.215,"15min":0.20699999999999999},"min":{"1min":0.17000000000000001,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.435,"5min":0.435,"15min":0.435},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.246,"15min":0.23899999999999999},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.25800000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.223},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.214},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.497},"last":0.27700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:39:23.888 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.888 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.888 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.888 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.888 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.889 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.889 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:23.889 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:39:24.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:24.151+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:24.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:24.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:25.188 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:25.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:25.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:25.250+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:26.155 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:26.154+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:26.209 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:26.209+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:27.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:27.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:27.208 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:27.207+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:28.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:28.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:28.205 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:28.205+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:29.191 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:29.190+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:29.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:29.236+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:30.163 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:30.162+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:30.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:30.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:31.137 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:31.136+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:31.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:31.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:32.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:32.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:32.184 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:32.184+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:33.177 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:33.176+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:33.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:33.219+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:34.142 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:34.141+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:34.219 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:34.218+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:35.117 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:35.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:35.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:35.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:36.148 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:36.147+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:36.266 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:36.265+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:37.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:37.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:37.241 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:37.240+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:38.157 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:38.156+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:38.263 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:38.262+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:39.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:39.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:39.254 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:39.254+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:40.183 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:40.183+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:40.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:40.302+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:41.171 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:41.171+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:41.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:41.340+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:42.194 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:42.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:42.300 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:42.299+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:43.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:43.228+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:43.351 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:43.350+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:43.890 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:39:44.209 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:39:44.211 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:39:44.222 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6000,"stamp":"2023-09-29T21:39:43.302170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002047"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3036,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:24.153622+0000","last_change":"2023-09-29T21:39:24.153622+0000","last_active":"2023-09-29T21:39:24.153622+0000","last_peered":"2023-09-29T21:39:24.153622+0000","last_clean":"2023-09-29T21:39:24.153622+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:39:24.153622+0000","last_undegraded":"2023-09-29T21:39:24.153622+0000","last_fullsized":"2023-09-29T21:39:24.153622+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12786,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:29.237198+0000","last_change":"2023-09-29T21:39:29.237198+0000","last_active":"2023-09-29T21:39:29.237198+0000","last_peered":"2023-09-29T21:39:29.237198+0000","last_clean":"2023-09-29T21:39:29.237198+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:39:29.237198+0000","last_undegraded":"2023-09-29T21:39:29.237198+0000","last_fullsized":"2023-09-29T21:39:29.237198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":340,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:30.254491+0000","last_change":"2023-09-29T21:39:30.254491+0000","last_active":"2023-09-29T21:39:30.254491+0000","last_peered":"2023-09-29T21:39:30.254491+0000","last_clean":"2023-09-29T21:39:30.254491+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:39:30.254491+0000","last_undegraded":"2023-09-29T21:39:30.254491+0000","last_fullsized":"2023-09-29T21:39:30.254491+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3524,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:32.169673+0000","last_change":"2023-09-29T21:39:32.169673+0000","last_active":"2023-09-29T21:39:32.169673+0000","last_peered":"2023-09-29T21:39:32.169673+0000","last_clean":"2023-09-29T21:39:32.169673+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:39:32.169673+0000","last_undegraded":"2023-09-29T21:39:32.169673+0000","last_fullsized":"2023-09-29T21:39:32.169673+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10862,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:26.155765+0000","last_change":"2023-09-29T21:39:26.155765+0000","last_active":"2023-09-29T21:39:26.155765+0000","last_peered":"2023-09-29T21:39:26.155765+0000","last_clean":"2023-09-29T21:39:26.155765+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:39:26.155765+0000","last_undegraded":"2023-09-29T21:39:26.155765+0000","last_fullsized":"2023-09-29T21:39:26.155765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3220,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:37.179381+0000","last_change":"2023-09-29T21:39:37.179381+0000","last_active":"2023-09-29T21:39:37.179381+0000","last_peered":"2023-09-29T21:39:37.179381+0000","last_clean":"2023-09-29T21:39:37.179381+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:39:37.179381+0000","last_undegraded":"2023-09-29T21:39:37.179381+0000","last_fullsized":"2023-09-29T21:39:37.179381+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740713,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.218},"min":{"1min":0.17000000000000001,"5min":0.16,"15min":0.154},"max":{"1min":0.33700000000000002,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25,"15min":0.23999999999999999},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40500000000000003,"5min":0.442,"15min":0.45900000000000002},"last":0.223}]},{"osd":1,"last update":"Fri Sep 29 21:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.253,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.74099999999999999,"15min":1.044},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27700000000000002,"15min":0.26400000000000001},"min":{"1min":0.21099999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.39600000000000002,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.26800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.20300000000000001,"15min":0.19900000000000001},"min":{"1min":0.153,"5min":0.14399999999999999,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.42099999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27000000000000002},"min":{"1min":0.21099999999999999,"5min":0.20799999999999999,"15min":0.17399999999999999},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"min":{"1min":0.183,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.251},"min":{"1min":0.20499999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.48899999999999999,"5min":0.504,"15min":0.504},"last":0.255}]}]},{"osd":1,"up_from":8,"seq":34359740710,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.21199999999999999,"15min":0.20499999999999999},"min":{"1min":0.14699999999999999,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.247,"5min":0.435,"15min":0.435},"last":0.191},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24299999999999999,"15min":0.23699999999999999},"min":{"1min":0.17599999999999999,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.28299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.224,"15min":0.221},"min":{"1min":0.17599999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.26000000000000001,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27400000000000002,"15min":0.27100000000000002},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.315,"5min":0.48899999999999999,"15min":0.497},"last":0.27100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:39:44.224 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.224 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.224 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.225 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.225 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.225 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.225 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:39:44.225 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:39:44.233 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:44.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:44.352 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:44.351+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:45.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:45.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:45.339 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:45.339+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:46.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:46.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:46.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:46.307+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:47.182 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:47.181+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:47.267 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:47.266+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:48.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:48.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:48.301 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:48.300+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:49.267 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:49.267+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:49.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:49.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:50.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:50.263+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:50.265 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:50.264+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:51.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:51.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:51.282 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:51.281+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:52.257 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:52.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:52.330 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:52.329+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:53.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:53.212+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:53.343 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:53.344+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:54.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:54.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:54.314 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:54.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:55.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:55.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:55.340 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:55.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:56.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:56.263+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:56.378 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:56.378+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:57.282 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:57.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:57.415 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:57.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:58.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:58.238+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:58.435 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:58.435+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:59.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:39:59.199+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:39:59.416 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:39:59.416+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:00.193 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:00.193+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:00.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:00.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:01.200 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:01.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:01.404 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:01.404+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:02.191 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:02.192+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:02.454 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:02.454+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:03.231 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:03.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:03.486 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:03.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:04.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:04.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:04.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:40:04.459 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:04.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:40:04.545 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:40:04.556 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6010,"stamp":"2023-09-29T21:40:03.305466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001896"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3039,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:54.187733+0000","last_change":"2023-09-29T21:39:54.187733+0000","last_active":"2023-09-29T21:39:54.187733+0000","last_peered":"2023-09-29T21:39:54.187733+0000","last_clean":"2023-09-29T21:39:54.187733+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:39:54.187733+0000","last_undegraded":"2023-09-29T21:39:54.187733+0000","last_fullsized":"2023-09-29T21:39:54.187733+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12789,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:59.200204+0000","last_change":"2023-09-29T21:39:59.200204+0000","last_active":"2023-09-29T21:39:59.200204+0000","last_peered":"2023-09-29T21:39:59.200204+0000","last_clean":"2023-09-29T21:39:59.200204+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:39:59.200204+0000","last_undegraded":"2023-09-29T21:39:59.200204+0000","last_fullsized":"2023-09-29T21:39:59.200204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":343,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:00.194041+0000","last_change":"2023-09-29T21:40:00.194041+0000","last_active":"2023-09-29T21:40:00.194041+0000","last_peered":"2023-09-29T21:40:00.194041+0000","last_clean":"2023-09-29T21:40:00.194041+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:40:00.194041+0000","last_undegraded":"2023-09-29T21:40:00.194041+0000","last_fullsized":"2023-09-29T21:40:00.194041+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3524,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:32.169673+0000","last_change":"2023-09-29T21:39:32.169673+0000","last_active":"2023-09-29T21:39:32.169673+0000","last_peered":"2023-09-29T21:39:32.169673+0000","last_clean":"2023-09-29T21:39:32.169673+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:39:32.169673+0000","last_undegraded":"2023-09-29T21:39:32.169673+0000","last_fullsized":"2023-09-29T21:39:32.169673+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10865,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:55.222705+0000","last_change":"2023-09-29T21:39:55.222705+0000","last_active":"2023-09-29T21:39:55.222705+0000","last_peered":"2023-09-29T21:39:55.222705+0000","last_clean":"2023-09-29T21:39:55.222705+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:39:55.222705+0000","last_undegraded":"2023-09-29T21:39:55.222705+0000","last_fullsized":"2023-09-29T21:39:55.222705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3220,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:39:37.179381+0000","last_change":"2023-09-29T21:39:37.179381+0000","last_active":"2023-09-29T21:39:37.179381+0000","last_peered":"2023-09-29T21:39:37.179381+0000","last_clean":"2023-09-29T21:39:37.179381+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:39:37.179381+0000","last_undegraded":"2023-09-29T21:39:37.179381+0000","last_fullsized":"2023-09-29T21:39:37.179381+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740717,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.218},"min":{"1min":0.17000000000000001,"5min":0.16,"15min":0.154},"max":{"1min":0.33700000000000002,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.184},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.25,"15min":0.23999999999999999},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40500000000000003,"5min":0.442,"15min":0.45900000000000002},"last":0.20499999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.253,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.74099999999999999,"15min":1.044},"last":0.222},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27700000000000002,"15min":0.26400000000000001},"min":{"1min":0.21099999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.39600000000000002,"5min":0.44900000000000001,"15min":0.50700000000000001},"last":0.29299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.19900000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.182},{"interface":"front","average":{"1min":0.255,"5min":0.27100000000000002,"15min":0.26700000000000002},"min":{"1min":0.183,"5min":0.183,"15min":0.17399999999999999},"max":{"1min":0.41799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.252}]},{"osd":2,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23400000000000001,"15min":0.22900000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.222},{"interface":"front","average":{"1min":0.253,"5min":0.255,"15min":0.25},"min":{"1min":0.20599999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"last":0.20799999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740714,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.21199999999999999,"15min":0.20499999999999999},"min":{"1min":0.14699999999999999,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.247,"5min":0.435,"15min":0.435},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24299999999999999,"15min":0.23699999999999999},"min":{"1min":0.17599999999999999,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.28299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.27900000000000003}]},{"osd":2,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.224,"15min":0.221},"min":{"1min":0.17599999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.26000000000000001,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.218},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27400000000000002,"15min":0.27100000000000002},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.315,"5min":0.48899999999999999,"15min":0.497},"last":0.29099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:40:04.557 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.558 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.558 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.558 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.558 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.559 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.559 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:04.559 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:40:05.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:05.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:05.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:05.427+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:06.276 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:06.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:06.432 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:06.432+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:07.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:07.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:07.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:07.393+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:08.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:08.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:08.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:08.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:09.237 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:09.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:09.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:09.402+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:10.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:10.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:10.353 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:10.354+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:11.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:11.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:11.363 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:11.363+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:12.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:12.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:12.397 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:12.397+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:13.153 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:13.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:13.398 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:13.398+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:14.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:14.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:14.396 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:14.396+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:15.209 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:15.210+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:15.382 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:15.382+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:16.197 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:16.197+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:16.377 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:16.377+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:17.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:17.187+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:17.341 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:17.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:18.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:18.152+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:18.307 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:18.308+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:19.111 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:19.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:19.273 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:19.273+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:20.068 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:20.068+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:20.278 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:20.278+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:21.037 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:21.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:21.269 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:21.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:22.087 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:22.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:22.251 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:22.251+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:23.049 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:23.049+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:23.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:23.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:24.029 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:24.029+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:24.245 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:24.245+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:24.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:40:24.874 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:40:24.876 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:40:24.887 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6020,"stamp":"2023-09-29T21:40:23.308689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001859"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3042,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:15.209928+0000","last_change":"2023-09-29T21:40:15.209928+0000","last_active":"2023-09-29T21:40:15.209928+0000","last_peered":"2023-09-29T21:40:15.209928+0000","last_clean":"2023-09-29T21:40:15.209928+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:40:15.209928+0000","last_undegraded":"2023-09-29T21:40:15.209928+0000","last_fullsized":"2023-09-29T21:40:15.209928+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12792,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:19.111300+0000","last_change":"2023-09-29T21:40:19.111300+0000","last_active":"2023-09-29T21:40:19.111300+0000","last_peered":"2023-09-29T21:40:19.111300+0000","last_clean":"2023-09-29T21:40:19.111300+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:40:19.111300+0000","last_undegraded":"2023-09-29T21:40:19.111300+0000","last_fullsized":"2023-09-29T21:40:19.111300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":346,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:20.068966+0000","last_change":"2023-09-29T21:40:20.068966+0000","last_active":"2023-09-29T21:40:20.068966+0000","last_peered":"2023-09-29T21:40:20.068966+0000","last_clean":"2023-09-29T21:40:20.068966+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:40:20.068966+0000","last_undegraded":"2023-09-29T21:40:20.068966+0000","last_fullsized":"2023-09-29T21:40:20.068966+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3530,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:21.038181+0000","last_change":"2023-09-29T21:40:21.038181+0000","last_active":"2023-09-29T21:40:21.038181+0000","last_peered":"2023-09-29T21:40:21.038181+0000","last_clean":"2023-09-29T21:40:21.038181+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:40:21.038181+0000","last_undegraded":"2023-09-29T21:40:21.038181+0000","last_fullsized":"2023-09-29T21:40:21.038181+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10868,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:18.152854+0000","last_change":"2023-09-29T21:40:18.152854+0000","last_active":"2023-09-29T21:40:18.152854+0000","last_peered":"2023-09-29T21:40:18.152854+0000","last_clean":"2023-09-29T21:40:18.152854+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:40:18.152854+0000","last_undegraded":"2023-09-29T21:40:18.152854+0000","last_fullsized":"2023-09-29T21:40:18.152854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3223,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:14.168638+0000","last_change":"2023-09-29T21:40:14.168638+0000","last_active":"2023-09-29T21:40:14.168638+0000","last_peered":"2023-09-29T21:40:14.168638+0000","last_clean":"2023-09-29T21:40:14.168638+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:40:14.168638+0000","last_undegraded":"2023-09-29T21:40:14.168638+0000","last_fullsized":"2023-09-29T21:40:14.168638+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740721,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.218},"min":{"1min":0.183,"5min":0.16,"15min":0.154},"max":{"1min":0.309,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.248,"15min":0.23999999999999999},"min":{"1min":0.17699999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34000000000000002,"5min":0.442,"15min":0.45900000000000002},"last":0.247}]},{"osd":1,"last update":"Fri Sep 29 21:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.246,"15min":0.24099999999999999},"min":{"1min":0.20399999999999999,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.36799999999999999,"5min":0.435,"15min":1.044},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.20699999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.37,"5min":0.44600000000000001,"15min":0.50700000000000001},"last":0.29099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.19900000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.27100000000000002,"15min":0.26700000000000002},"min":{"1min":0.183,"5min":0.183,"15min":0.17399999999999999},"max":{"1min":0.41799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.17000000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23400000000000001,"15min":0.22900000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.253,"5min":0.255,"15min":0.25},"min":{"1min":0.20599999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"last":0.22900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740718,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.21199999999999999,"15min":0.20499999999999999},"min":{"1min":0.14699999999999999,"5min":0.13900000000000001,"15min":0.114},"max":{"1min":0.247,"5min":0.435,"15min":0.435},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24299999999999999,"15min":0.23699999999999999},"min":{"1min":0.17599999999999999,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.28299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.17299999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:39:35 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.224,"15min":0.221},"min":{"1min":0.17599999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.26000000000000001,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27400000000000002,"15min":0.27100000000000002},"min":{"1min":0.22800000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.315,"5min":0.48899999999999999,"15min":0.497},"last":0.247}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:40:24.889 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.890 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.890 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.890 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.890 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.891 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.891 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:24.891 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:40:25.074 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:25.074+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:25.197 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:25.197+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:26.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:26.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:26.233 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:26.233+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:27.050 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:27.051+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:27.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:27.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:28.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:28.005+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:28.287 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:28.287+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:28.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:28.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:29.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:29.317+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:29.926 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:29.926+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:30.365 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:30.365+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:30.947 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:30.946+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:31.371 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:31.371+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:31.918 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:31.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:32.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:32.322+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:32.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:32.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:33.303 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:33.303+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:33.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:33.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:34.285 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:34.285+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:34.935 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:34.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:35.261 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:35.261+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:35.920 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:35.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:36.268 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:36.269+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:36.892 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:36.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:37.255 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:37.255+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:37.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:37.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:38.213 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:38.213+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:38.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:38.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:39.249 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:39.249+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:39.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:39.853+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:40.275 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:40.275+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:40.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:40.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:41.297 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:41.297+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:41.793 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:41.794+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:42.304 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:42.304+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:42.757 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:42.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:43.311 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:43.310+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:43.767 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:43.767+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:44.319 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:44.319+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:44.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:44.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:44.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:40:45.209 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:40:45.212 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:40:45.224 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6030,"stamp":"2023-09-29T21:40:43.311735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001917"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3045,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:27.050968+0000","last_change":"2023-09-29T21:40:27.050968+0000","last_active":"2023-09-29T21:40:27.050968+0000","last_peered":"2023-09-29T21:40:27.050968+0000","last_clean":"2023-09-29T21:40:27.050968+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:40:27.050968+0000","last_undegraded":"2023-09-29T21:40:27.050968+0000","last_fullsized":"2023-09-29T21:40:27.050968+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12795,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:30.946977+0000","last_change":"2023-09-29T21:40:30.946977+0000","last_active":"2023-09-29T21:40:30.946977+0000","last_peered":"2023-09-29T21:40:30.946977+0000","last_clean":"2023-09-29T21:40:30.946977+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:40:30.946977+0000","last_undegraded":"2023-09-29T21:40:30.946977+0000","last_fullsized":"2023-09-29T21:40:30.946977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":349,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:31.918892+0000","last_change":"2023-09-29T21:40:31.918892+0000","last_active":"2023-09-29T21:40:31.918892+0000","last_peered":"2023-09-29T21:40:31.918892+0000","last_clean":"2023-09-29T21:40:31.918892+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:40:31.918892+0000","last_undegraded":"2023-09-29T21:40:31.918892+0000","last_fullsized":"2023-09-29T21:40:31.918892+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3533,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:33.911804+0000","last_change":"2023-09-29T21:40:33.911804+0000","last_active":"2023-09-29T21:40:33.911804+0000","last_peered":"2023-09-29T21:40:33.911804+0000","last_clean":"2023-09-29T21:40:33.911804+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:40:33.911804+0000","last_undegraded":"2023-09-29T21:40:33.911804+0000","last_fullsized":"2023-09-29T21:40:33.911804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10871,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:29.926860+0000","last_change":"2023-09-29T21:40:29.926860+0000","last_active":"2023-09-29T21:40:29.926860+0000","last_peered":"2023-09-29T21:40:29.926860+0000","last_clean":"2023-09-29T21:40:29.926860+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:40:29.926860+0000","last_undegraded":"2023-09-29T21:40:29.926860+0000","last_fullsized":"2023-09-29T21:40:29.926860+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3226,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:23.050049+0000","last_change":"2023-09-29T21:40:23.050049+0000","last_active":"2023-09-29T21:40:23.050049+0000","last_peered":"2023-09-29T21:40:23.050049+0000","last_clean":"2023-09-29T21:40:23.050049+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:40:23.050049+0000","last_undegraded":"2023-09-29T21:40:23.050049+0000","last_fullsized":"2023-09-29T21:40:23.050049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740725,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.218},"min":{"1min":0.183,"5min":0.16,"15min":0.154},"max":{"1min":0.309,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.248,"15min":0.23999999999999999},"min":{"1min":0.17699999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34000000000000002,"5min":0.442,"15min":0.45900000000000002},"last":0.216}]},{"osd":1,"last update":"Fri Sep 29 21:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.246,"15min":0.24099999999999999},"min":{"1min":0.20399999999999999,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.36799999999999999,"5min":0.435,"15min":1.044},"last":0.252},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.20699999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.37,"5min":0.44600000000000001,"15min":0.50700000000000001},"last":0.246}]}]},{"osd":0,"up_from":8,"seq":34359740724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.19900000000000001},"min":{"1min":0.16700000000000001,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.27100000000000002,"15min":0.26700000000000002},"min":{"1min":0.183,"5min":0.183,"15min":0.17399999999999999},"max":{"1min":0.41799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.21099999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23400000000000001,"15min":0.22900000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.253,"5min":0.255,"15min":0.25},"min":{"1min":0.20599999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"last":0.22500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740722,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.16700000000000001,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.39100000000000001,"5min":0.435,"15min":0.435},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17100000000000001,"15min":0.13600000000000001},"max":{"1min":0.42399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.20599999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.221,"15min":0.22},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.40100000000000002,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.187},{"interface":"front","average":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.442,"5min":0.48899999999999999,"15min":0.497},"last":0.24099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:40:45.225 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.225 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.226 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.226 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.226 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.226 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.227 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:40:45.227 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:40:45.368 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:45.368+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:45.689 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:45.689+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:46.333 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:46.333+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:46.731 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:46.730+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:47.332 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:47.332+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:47.742 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:47.742+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:48.328 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:48.328+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:48.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:48.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:49.334 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:49.334+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:49.714 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:49.713+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:50.322 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:50.321+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:50.734 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:50.733+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:51.313 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:51.312+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:51.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:51.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:52.277 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:52.276+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:52.751 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:52.750+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:53.316 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:53.315+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:53.736 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:53.735+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:54.327 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:54.326+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:54.755 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:54.754+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:55.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:55.341+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:55.739 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:55.738+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:56.384 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:56.384+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:56.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:56.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:57.386 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:57.385+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:57.796 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:57.796+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:58.427 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:58.426+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:58.747 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:58.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:59.409 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:40:59.408+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:40:59.717 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:40:59.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:00.446 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:00.445+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:00.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:00.715+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:01.422 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:01.421+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:01.692 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:01.691+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:02.375 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:02.374+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:02.708 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:02.706+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:03.414 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:03.413+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:03.720 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:03.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:04.463 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:04.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:04.703 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:04.703+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:05.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:41:05.431 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:05.430+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:05.563 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:41:05.565 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:41:05.577 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6041,"stamp":"2023-09-29T21:41:05.314955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001699"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3051,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:01.692755+0000","last_change":"2023-09-29T21:41:01.692755+0000","last_active":"2023-09-29T21:41:01.692755+0000","last_peered":"2023-09-29T21:41:01.692755+0000","last_clean":"2023-09-29T21:41:01.692755+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:41:01.692755+0000","last_undegraded":"2023-09-29T21:41:01.692755+0000","last_fullsized":"2023-09-29T21:41:01.692755+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12798,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:50.734430+0000","last_change":"2023-09-29T21:40:50.734430+0000","last_active":"2023-09-29T21:40:50.734430+0000","last_peered":"2023-09-29T21:40:50.734430+0000","last_clean":"2023-09-29T21:40:50.734430+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:40:50.734430+0000","last_undegraded":"2023-09-29T21:40:50.734430+0000","last_fullsized":"2023-09-29T21:40:50.734430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":352,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:52.751244+0000","last_change":"2023-09-29T21:40:52.751244+0000","last_active":"2023-09-29T21:40:52.751244+0000","last_peered":"2023-09-29T21:40:52.751244+0000","last_clean":"2023-09-29T21:40:52.751244+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:40:52.751244+0000","last_undegraded":"2023-09-29T21:40:52.751244+0000","last_fullsized":"2023-09-29T21:40:52.751244+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3536,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:53.737099+0000","last_change":"2023-09-29T21:40:53.737099+0000","last_active":"2023-09-29T21:40:53.737099+0000","last_peered":"2023-09-29T21:40:53.737099+0000","last_clean":"2023-09-29T21:40:53.737099+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:40:53.737099+0000","last_undegraded":"2023-09-29T21:40:53.737099+0000","last_fullsized":"2023-09-29T21:40:53.737099+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10874,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:46.731292+0000","last_change":"2023-09-29T21:40:46.731292+0000","last_active":"2023-09-29T21:40:46.731292+0000","last_peered":"2023-09-29T21:40:46.731292+0000","last_clean":"2023-09-29T21:40:46.731292+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:40:46.731292+0000","last_undegraded":"2023-09-29T21:40:46.731292+0000","last_fullsized":"2023-09-29T21:40:46.731292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12042,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:36:58.718749+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:36:58.718749+0000","last_peered":"2023-09-29T21:36:58.718749+0000","last_clean":"2023-09-29T21:36:58.718749+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:36:58.718749+0000","last_undegraded":"2023-09-29T21:36:58.718749+0000","last_fullsized":"2023-09-29T21:36:58.718749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 267s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3232,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:40:54.755735+0000","last_change":"2023-09-29T21:40:54.755735+0000","last_active":"2023-09-29T21:40:54.755735+0000","last_peered":"2023-09-29T21:40:54.755735+0000","last_clean":"2023-09-29T21:40:54.755735+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:40:54.755735+0000","last_undegraded":"2023-09-29T21:40:54.755735+0000","last_fullsized":"2023-09-29T21:40:54.755735+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740729,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.218},"min":{"1min":0.183,"5min":0.16,"15min":0.154},"max":{"1min":0.309,"5min":0.38900000000000001,"15min":0.48999999999999999},"last":0.191},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.248,"15min":0.23999999999999999},"min":{"1min":0.17699999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.34000000000000002,"5min":0.442,"15min":0.45900000000000002},"last":0.22700000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.246,"15min":0.24099999999999999},"min":{"1min":0.20399999999999999,"5min":0.185,"15min":0.17299999999999999},"max":{"1min":0.36799999999999999,"5min":0.435,"15min":1.044},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.27100000000000002,"15min":0.26300000000000001},"min":{"1min":0.20699999999999999,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.37,"5min":0.44600000000000001,"15min":0.50700000000000001},"last":0.27800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.20100000000000001},"min":{"1min":0.182,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.36399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22900000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.252,"15min":0.25},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40200000000000002,"5min":0.48899999999999999,"15min":0.504},"last":0.25}]}]},{"osd":1,"up_from":8,"seq":34359740726,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.16700000000000001,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.39100000000000001,"5min":0.435,"15min":0.435},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17100000000000001,"15min":0.13600000000000001},"max":{"1min":0.42399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.20599999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.221,"15min":0.22},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.40100000000000002,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.187},{"interface":"front","average":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.442,"5min":0.48899999999999999,"15min":0.497},"last":0.23200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:41:05.579 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.579 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.579 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.579 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.579 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.580 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.580 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:05.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:41:05.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:05.656+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:05.923 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:41:05.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:41:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:41:06.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:41:06.407 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:06.406+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:41:06.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:41:06.695 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:06.694+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:06.930 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:41:06.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:41:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:41:07.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:41:07.376 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:07.375+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:07.644 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:41:07.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:41:07.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:07.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:41:08.013 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:41:08.420 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:08.419+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:08.701 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:08.701+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:09.393 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:09.392+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:09.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:09.727+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:10.410 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:10.409+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:10.758 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:10.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:11.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:11.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:11.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:11.736+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:12.430 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:12.429+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:12.715 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:12.714+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:13.401 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:13.400+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:13.735 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:13.734+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:14.442 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:14.442+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:14.726 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:14.726+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:15.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:15.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:15.716 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:15.716+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:16.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:16.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:16.669 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:16.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:17.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:17.509+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:17.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:17.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:18.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:18.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:18.600 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:18.599+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:19.571 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:19.570+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:19.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:19.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:20.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:20.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:20.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:20.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:21.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:21.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:21.654 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:21.654+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:22.515 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:22.515+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:22.627 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:22.626+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:23.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:23.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:23.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:23.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:24.513 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:24.513+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:24.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:24.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:25.499 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:25.498+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:25.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:25.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:26.477 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:26.477+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:26.601 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:26.600+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:27.523 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:27.522+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:27.636 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:27.636+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:28.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:41:28.340 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:41:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:41:28.352 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6052,"stamp":"2023-09-29T21:41:27.317992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001622"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3055,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:13.735692+0000","last_change":"2023-09-29T21:41:13.735692+0000","last_active":"2023-09-29T21:41:13.735692+0000","last_peered":"2023-09-29T21:41:13.735692+0000","last_clean":"2023-09-29T21:41:13.735692+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:41:13.735692+0000","last_undegraded":"2023-09-29T21:41:13.735692+0000","last_fullsized":"2023-09-29T21:41:13.735692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12805,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:18.600696+0000","last_change":"2023-09-29T21:41:18.600696+0000","last_active":"2023-09-29T21:41:18.600696+0000","last_peered":"2023-09-29T21:41:18.600696+0000","last_clean":"2023-09-29T21:41:18.600696+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:41:18.600696+0000","last_undegraded":"2023-09-29T21:41:18.600696+0000","last_fullsized":"2023-09-29T21:41:18.600696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":359,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:19.651235+0000","last_change":"2023-09-29T21:41:19.651235+0000","last_active":"2023-09-29T21:41:19.651235+0000","last_peered":"2023-09-29T21:41:19.651235+0000","last_clean":"2023-09-29T21:41:19.651235+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:41:19.651235+0000","last_undegraded":"2023-09-29T21:41:19.651235+0000","last_fullsized":"2023-09-29T21:41:19.651235+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3540,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:09.728793+0000","last_change":"2023-09-29T21:41:09.728793+0000","last_active":"2023-09-29T21:41:09.728793+0000","last_peered":"2023-09-29T21:41:09.728793+0000","last_clean":"2023-09-29T21:41:09.728793+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:41:09.728793+0000","last_undegraded":"2023-09-29T21:41:09.728793+0000","last_fullsized":"2023-09-29T21:41:09.728793+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10881,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:11.737638+0000","last_change":"2023-09-29T21:41:11.737638+0000","last_active":"2023-09-29T21:41:11.737638+0000","last_peered":"2023-09-29T21:41:11.737638+0000","last_clean":"2023-09-29T21:41:11.737638+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:41:11.737638+0000","last_undegraded":"2023-09-29T21:41:11.737638+0000","last_fullsized":"2023-09-29T21:41:11.737638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3236,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:14.727508+0000","last_change":"2023-09-29T21:41:14.727508+0000","last_active":"2023-09-29T21:41:14.727508+0000","last_peered":"2023-09-29T21:41:14.727508+0000","last_clean":"2023-09-29T21:41:14.727508+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:41:14.727508+0000","last_undegraded":"2023-09-29T21:41:14.727508+0000","last_fullsized":"2023-09-29T21:41:14.727508+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740733,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.22800000000000001,"15min":0.219},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.17299999999999999},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.33000000000000002,"5min":0.40500000000000003,"15min":0.44900000000000001},"last":0.188}]},{"osd":1,"last update":"Fri Sep 29 21:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17299999999999999},"max":{"1min":0.28399999999999997,"5min":0.40100000000000002,"15min":1.044},"last":0.246},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26800000000000002,"15min":0.26300000000000001},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.372,"5min":0.443,"15min":0.50700000000000001},"last":0.215}]}]},{"osd":0,"up_from":8,"seq":34359740732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.20100000000000001},"min":{"1min":0.182,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.36399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.23400000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22900000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.252,"15min":0.25},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40200000000000002,"5min":0.48899999999999999,"15min":0.504},"last":0.19700000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740730,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.16700000000000001,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.39100000000000001,"5min":0.435,"15min":0.435},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.17100000000000001,"15min":0.13600000000000001},"max":{"1min":0.42399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.221,"15min":0.22},"min":{"1min":0.17899999999999999,"5min":0.155,"15min":0.155},"max":{"1min":0.40100000000000002,"5min":0.42199999999999999,"15min":0.44700000000000001},"last":0.214},{"interface":"front","average":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.27100000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.442,"5min":0.48899999999999999,"15min":0.497},"last":0.27400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:41:28.354 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.354 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.354 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.355 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.355 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.355 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.355 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:28.355 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:41:28.553 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:28.552+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:28.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:28.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:29.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:29.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:29.686 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:29.686+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:30.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:30.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:30.650 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:30.650+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:31.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:31.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:31.628 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:31.628+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:32.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:32.564+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:32.625 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:32.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:33.594 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:33.594+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:33.642 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:33.642+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:34.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:34.553+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:34.671 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:34.671+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:35.516 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:35.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:35.626 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:35.625+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:36.521 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:36.520+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:36.609 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:36.608+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:37.518 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:37.517+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:37.573 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:37.572+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:38.536 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:38.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:38.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:38.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:39.551 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:39.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:39.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:39.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:40.589 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:40.588+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:40.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:40.607+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:41.585 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:41.584+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:41.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:41.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:42.613 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:42.612+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:42.617 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:42.616+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:43.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:43.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:43.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:43.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:44.601 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:44.600+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:44.648 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:44.648+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:45.605 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:45.604+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:45.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:45.649+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:46.557 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:46.556+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:46.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:46.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:47.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:47.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:47.670 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:47.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:48.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:41:48.567 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:48.566+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:48.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:48.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:48.673 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:41:48.675 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:41:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6062,"stamp":"2023-09-29T21:41:47.320790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001678"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3058,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:36.609290+0000","last_change":"2023-09-29T21:41:36.609290+0000","last_active":"2023-09-29T21:41:36.609290+0000","last_peered":"2023-09-29T21:41:36.609290+0000","last_clean":"2023-09-29T21:41:36.609290+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:41:36.609290+0000","last_undegraded":"2023-09-29T21:41:36.609290+0000","last_fullsized":"2023-09-29T21:41:36.609290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12808,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:38.536958+0000","last_change":"2023-09-29T21:41:38.536958+0000","last_active":"2023-09-29T21:41:38.536958+0000","last_peered":"2023-09-29T21:41:38.536958+0000","last_clean":"2023-09-29T21:41:38.536958+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:41:38.536958+0000","last_undegraded":"2023-09-29T21:41:38.536958+0000","last_fullsized":"2023-09-29T21:41:38.536958+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":362,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:40.589640+0000","last_change":"2023-09-29T21:41:40.589640+0000","last_active":"2023-09-29T21:41:40.589640+0000","last_peered":"2023-09-29T21:41:40.589640+0000","last_clean":"2023-09-29T21:41:40.589640+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:41:40.589640+0000","last_undegraded":"2023-09-29T21:41:40.589640+0000","last_fullsized":"2023-09-29T21:41:40.589640+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3543,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:28.658283+0000","last_change":"2023-09-29T21:41:28.658283+0000","last_active":"2023-09-29T21:41:28.658283+0000","last_peered":"2023-09-29T21:41:28.658283+0000","last_clean":"2023-09-29T21:41:28.658283+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:41:28.658283+0000","last_undegraded":"2023-09-29T21:41:28.658283+0000","last_fullsized":"2023-09-29T21:41:28.658283+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10884,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:31.629344+0000","last_change":"2023-09-29T21:41:31.629344+0000","last_active":"2023-09-29T21:41:31.629344+0000","last_peered":"2023-09-29T21:41:31.629344+0000","last_clean":"2023-09-29T21:41:31.629344+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:41:31.629344+0000","last_undegraded":"2023-09-29T21:41:31.629344+0000","last_fullsized":"2023-09-29T21:41:31.629344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3239,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:37.573731+0000","last_change":"2023-09-29T21:41:37.573731+0000","last_active":"2023-09-29T21:41:37.573731+0000","last_peered":"2023-09-29T21:41:37.573731+0000","last_clean":"2023-09-29T21:41:37.573731+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:41:37.573731+0000","last_undegraded":"2023-09-29T21:41:37.573731+0000","last_fullsized":"2023-09-29T21:41:37.573731+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740737,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.22800000000000001,"15min":0.219},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.33000000000000002,"5min":0.40500000000000003,"15min":0.44900000000000001},"last":0.223}]},{"osd":1,"last update":"Fri Sep 29 21:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17299999999999999},"max":{"1min":0.28399999999999997,"5min":0.40100000000000002,"15min":1.044},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26800000000000002,"15min":0.26300000000000001},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.372,"5min":0.443,"15min":0.50700000000000001},"last":0.26000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.20100000000000001},"min":{"1min":0.182,"5min":0.14899999999999999,"15min":0.13300000000000001},"max":{"1min":0.36399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.248}]},{"osd":2,"last update":"Fri Sep 29 21:40:53 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.22900000000000001},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.252,"15min":0.25},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.40200000000000002,"5min":0.48899999999999999,"15min":0.504},"last":0.22}]}]},{"osd":1,"up_from":8,"seq":34359740734,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20699999999999999,"15min":0.20399999999999999},"min":{"1min":0.157,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.35099999999999998,"5min":0.435,"15min":0.435},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.19,"5min":0.17299999999999999,"15min":0.13600000000000001},"max":{"1min":0.29999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.254}]},{"osd":2,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.16900000000000001,"5min":0.155,"15min":0.155},"max":{"1min":0.25600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.32400000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.24199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:41:48.688 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.688 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.688 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.689 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.689 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.689 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.689 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:41:48.689 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:41:49.602 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:49.602+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:49.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:49.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:50.578 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:50.578+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:50.676 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:50.675+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:51.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:51.586+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:51.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:51.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:52.591 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:52.592+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:52.729 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:52.729+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:53.629 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:53.629+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:53.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:53.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:54.645 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:54.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:54.686 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:54.686+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:55.646 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:55.646+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:55.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:55.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:56.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:56.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:56.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:56.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:57.672 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:57.672+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:57.690 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:57.690+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:58.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:58.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:58.681 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:58.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:59.624 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:41:59.624+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:41:59.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:41:59.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:00.582 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:00.582+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:00.619 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:00.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:01.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:01.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:01.621 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:01.621+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:02.603 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:02.604+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:02.670 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:02.670+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:03.629 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:03.630+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:03.680 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:03.680+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:04.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:04.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:04.668 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:04.668+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:05.563 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:05.563+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:05.718 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:05.718+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:06.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:06.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:06.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:06.674+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:07.512 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:07.512+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:07.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:07.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:08.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:08.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:08.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:42:08.728 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:08.729+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:09.023 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:42:09.025 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:42:09.036 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6072,"stamp":"2023-09-29T21:42:07.323861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001901"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3061,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:47.532236+0000","last_change":"2023-09-29T21:41:47.532236+0000","last_active":"2023-09-29T21:41:47.532236+0000","last_peered":"2023-09-29T21:41:47.532236+0000","last_clean":"2023-09-29T21:41:47.532236+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:41:47.532236+0000","last_undegraded":"2023-09-29T21:41:47.532236+0000","last_fullsized":"2023-09-29T21:41:47.532236+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12811,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:49.603347+0000","last_change":"2023-09-29T21:41:49.603347+0000","last_active":"2023-09-29T21:41:49.603347+0000","last_peered":"2023-09-29T21:41:49.603347+0000","last_clean":"2023-09-29T21:41:49.603347+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:41:49.603347+0000","last_undegraded":"2023-09-29T21:41:49.603347+0000","last_fullsized":"2023-09-29T21:41:49.603347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":365,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:50.579013+0000","last_change":"2023-09-29T21:41:50.579013+0000","last_active":"2023-09-29T21:41:50.579013+0000","last_peered":"2023-09-29T21:41:50.579013+0000","last_clean":"2023-09-29T21:41:50.579013+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:41:50.579013+0000","last_undegraded":"2023-09-29T21:41:50.579013+0000","last_fullsized":"2023-09-29T21:41:50.579013+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3549,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:58.658513+0000","last_change":"2023-09-29T21:41:58.658513+0000","last_active":"2023-09-29T21:41:58.658513+0000","last_peered":"2023-09-29T21:41:58.658513+0000","last_clean":"2023-09-29T21:41:58.658513+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:41:58.658513+0000","last_undegraded":"2023-09-29T21:41:58.658513+0000","last_fullsized":"2023-09-29T21:41:58.658513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10890,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:59.624393+0000","last_change":"2023-09-29T21:41:59.624393+0000","last_active":"2023-09-29T21:41:59.624393+0000","last_peered":"2023-09-29T21:41:59.624393+0000","last_clean":"2023-09-29T21:41:59.624393+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:41:59.624393+0000","last_undegraded":"2023-09-29T21:41:59.624393+0000","last_fullsized":"2023-09-29T21:41:59.624393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3242,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:41:48.567691+0000","last_change":"2023-09-29T21:41:48.567691+0000","last_active":"2023-09-29T21:41:48.567691+0000","last_peered":"2023-09-29T21:41:48.567691+0000","last_clean":"2023-09-29T21:41:48.567691+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:41:48.567691+0000","last_undegraded":"2023-09-29T21:41:48.567691+0000","last_fullsized":"2023-09-29T21:41:48.567691+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740741,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.22800000000000001,"15min":0.219},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.191},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"min":{"1min":0.153,"5min":0.153,"15min":0.153},"max":{"1min":0.33000000000000002,"5min":0.40500000000000003,"15min":0.44900000000000001},"last":0.245}]},{"osd":1,"last update":"Fri Sep 29 21:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17299999999999999},"max":{"1min":0.28399999999999997,"5min":0.40100000000000002,"15min":1.044},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26800000000000002,"15min":0.26300000000000001},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.17799999999999999},"max":{"1min":0.372,"5min":0.443,"15min":0.50700000000000001},"last":0.27600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.221,"15min":0.20300000000000001},"min":{"1min":0.157,"5min":0.153,"15min":0.13300000000000001},"max":{"1min":0.32200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.29099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.30099999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.23200000000000001,"15min":0.22800000000000001},"min":{"1min":0.17100000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.30499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.218},{"interface":"front","average":{"1min":0.249,"5min":0.253,"15min":0.249},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.42099999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.28599999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740738,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20699999999999999,"15min":0.20399999999999999},"min":{"1min":0.157,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.35099999999999998,"5min":0.435,"15min":0.435},"last":0.14599999999999999},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.19,"5min":0.17299999999999999,"15min":0.13600000000000001},"max":{"1min":0.29999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.17000000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.16900000000000001,"5min":0.155,"15min":0.155},"max":{"1min":0.25600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.32400000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.23000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:42:09.037 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.038 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.038 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.038 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.038 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.039 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.039 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:09.039 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:42:09.489 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:09.490+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:09.719 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:09.719+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:10.537 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:10.538+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:10.724 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:10.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:11.555 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:11.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:11.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:11.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:12.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:12.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:12.808 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:12.808+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:13.591 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:13.591+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:13.850 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:13.850+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:14.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:14.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:14.886 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:14.886+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:15.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:15.575+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:15.839 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:15.839+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:16.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:16.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:16.822 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:16.823+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:17.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:17.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:17.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:17.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:18.527 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:18.528+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:18.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:18.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:19.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:19.533+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:19.916 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:19.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:20.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:20.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:20.890 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:20.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:21.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:21.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:21.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:21.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:22.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:22.518+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:22.860 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:22.861+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:23.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:23.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:23.859 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:23.859+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:24.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:24.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:24.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:24.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:25.554 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:25.554+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:25.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:25.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:26.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:26.590+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:26.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:26.870+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:27.572 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:27.572+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:27.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:27.918+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:28.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:28.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:28.930 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:28.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:29.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:42:29.359 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:42:29.362 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:42:29.372 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6083,"stamp":"2023-09-29T21:42:29.327322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001814"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3067,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:19.916788+0000","last_change":"2023-09-29T21:42:19.916788+0000","last_active":"2023-09-29T21:42:19.916788+0000","last_peered":"2023-09-29T21:42:19.916788+0000","last_clean":"2023-09-29T21:42:19.916788+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:42:19.916788+0000","last_undegraded":"2023-09-29T21:42:19.916788+0000","last_fullsized":"2023-09-29T21:42:19.916788+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12817,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:23.859760+0000","last_change":"2023-09-29T21:42:23.859760+0000","last_active":"2023-09-29T21:42:23.859760+0000","last_peered":"2023-09-29T21:42:23.859760+0000","last_clean":"2023-09-29T21:42:23.859760+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:42:23.859760+0000","last_undegraded":"2023-09-29T21:42:23.859760+0000","last_fullsized":"2023-09-29T21:42:23.859760+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":368,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:16.823052+0000","last_change":"2023-09-29T21:42:16.823052+0000","last_active":"2023-09-29T21:42:16.823052+0000","last_peered":"2023-09-29T21:42:16.823052+0000","last_clean":"2023-09-29T21:42:16.823052+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:42:16.823052+0000","last_undegraded":"2023-09-29T21:42:16.823052+0000","last_fullsized":"2023-09-29T21:42:16.823052+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3552,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:17.866570+0000","last_change":"2023-09-29T21:42:17.866570+0000","last_active":"2023-09-29T21:42:17.866570+0000","last_peered":"2023-09-29T21:42:17.866570+0000","last_clean":"2023-09-29T21:42:17.866570+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:42:17.866570+0000","last_undegraded":"2023-09-29T21:42:17.866570+0000","last_fullsized":"2023-09-29T21:42:17.866570+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10893,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:18.882283+0000","last_change":"2023-09-29T21:42:18.882283+0000","last_active":"2023-09-29T21:42:18.882283+0000","last_peered":"2023-09-29T21:42:18.882283+0000","last_clean":"2023-09-29T21:42:18.882283+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:42:18.882283+0000","last_undegraded":"2023-09-29T21:42:18.882283+0000","last_fullsized":"2023-09-29T21:42:18.882283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3248,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:22.861271+0000","last_change":"2023-09-29T21:42:22.861271+0000","last_active":"2023-09-29T21:42:22.861271+0000","last_peered":"2023-09-29T21:42:22.861271+0000","last_clean":"2023-09-29T21:42:22.861271+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:42:22.861271+0000","last_undegraded":"2023-09-29T21:42:22.861271+0000","last_fullsized":"2023-09-29T21:42:22.861271+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740746,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.223,"15min":0.217},"min":{"1min":0.159,"5min":0.151,"15min":0.151},"max":{"1min":0.24299999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.21299999999999999,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.25600000000000001,"5min":0.40500000000000003,"15min":0.442},"last":0.23699999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:42:09 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.23599999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.27800000000000002,"5min":0.372,"15min":1.044},"last":0.224},{"interface":"front","average":{"1min":0.25,"5min":0.26300000000000001,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.30199999999999999,"5min":0.39600000000000002,"15min":0.44900000000000001},"last":0.19600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.221,"15min":0.20300000000000001},"min":{"1min":0.157,"5min":0.153,"15min":0.13300000000000001},"max":{"1min":0.32200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.29099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.245}]},{"osd":2,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.23200000000000001,"15min":0.22800000000000001},"min":{"1min":0.17100000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.30499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.253,"15min":0.249},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.42099999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.26500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740743,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.19600000000000001,"5min":0.20699999999999999,"15min":0.20399999999999999},"min":{"1min":0.157,"5min":0.14699999999999999,"15min":0.114},"max":{"1min":0.35099999999999998,"5min":0.435,"15min":0.435},"last":0.189},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23599999999999999,"15min":0.23499999999999999},"min":{"1min":0.19,"5min":0.17299999999999999,"15min":0.13600000000000001},"max":{"1min":0.29999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:41:39 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.16900000000000001,"5min":0.155,"15min":0.155},"max":{"1min":0.25600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"min":{"1min":0.218,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.32400000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:42:29.374 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.375 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:29.376 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:42:29.503 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:29.503+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:29.881 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:29.882+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:30.526 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:30.526+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:30.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:30.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:31.536 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:31.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:31.897 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:31.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:32.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:32.525+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:32.871 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:32.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:33.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:33.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:33.854 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:33.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:34.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:34.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:34.887 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:34.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:35.573 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:35.573+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:35.891 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:35.891+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:36.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:36.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:36.896 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:36.896+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:37.593 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:37.593+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:37.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:37.866+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:38.637 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:38.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:38.832 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:38.832+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:39.612 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:39.612+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:39.833 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:39.833+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:40.574 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:40.574+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:40.851 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:40.852+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:41.544 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:41.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:41.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:41.856+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:42.540 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:42.540+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:42.824 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:42.825+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:43.511 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:43.511+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:43.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:43.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:44.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:44.509+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:44.877 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:44.877+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:45.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:45.505+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:45.875 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:45.875+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:46.472 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:46.471+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:46.921 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:46.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:47.460 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:47.459+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:47.952 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:47.951+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:48.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:48.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:48.957 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:48.956+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:49.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:42:49.495 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:49.494+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:49.695 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:42:49.697 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:42:49.708 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6093,"stamp":"2023-09-29T21:42:49.330580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002086"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3070,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:38.832683+0000","last_change":"2023-09-29T21:42:38.832683+0000","last_active":"2023-09-29T21:42:38.832683+0000","last_peered":"2023-09-29T21:42:38.832683+0000","last_clean":"2023-09-29T21:42:38.832683+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:42:38.832683+0000","last_undegraded":"2023-09-29T21:42:38.832683+0000","last_fullsized":"2023-09-29T21:42:38.832683+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12820,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:45.876320+0000","last_change":"2023-09-29T21:42:45.876320+0000","last_active":"2023-09-29T21:42:45.876320+0000","last_peered":"2023-09-29T21:42:45.876320+0000","last_clean":"2023-09-29T21:42:45.876320+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:42:45.876320+0000","last_undegraded":"2023-09-29T21:42:45.876320+0000","last_fullsized":"2023-09-29T21:42:45.876320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":371,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:32.872088+0000","last_change":"2023-09-29T21:42:32.872088+0000","last_active":"2023-09-29T21:42:32.872088+0000","last_peered":"2023-09-29T21:42:32.872088+0000","last_clean":"2023-09-29T21:42:32.872088+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:42:32.872088+0000","last_undegraded":"2023-09-29T21:42:32.872088+0000","last_fullsized":"2023-09-29T21:42:32.872088+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3555,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:33.855434+0000","last_change":"2023-09-29T21:42:33.855434+0000","last_active":"2023-09-29T21:42:33.855434+0000","last_peered":"2023-09-29T21:42:33.855434+0000","last_clean":"2023-09-29T21:42:33.855434+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:42:33.855434+0000","last_undegraded":"2023-09-29T21:42:33.855434+0000","last_fullsized":"2023-09-29T21:42:33.855434+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10896,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:34.888475+0000","last_change":"2023-09-29T21:42:34.888475+0000","last_active":"2023-09-29T21:42:34.888475+0000","last_peered":"2023-09-29T21:42:34.888475+0000","last_clean":"2023-09-29T21:42:34.888475+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:42:34.888475+0000","last_undegraded":"2023-09-29T21:42:34.888475+0000","last_fullsized":"2023-09-29T21:42:34.888475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3251,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:44.878375+0000","last_change":"2023-09-29T21:42:44.878375+0000","last_active":"2023-09-29T21:42:44.878375+0000","last_peered":"2023-09-29T21:42:44.878375+0000","last_clean":"2023-09-29T21:42:44.878375+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:42:44.878375+0000","last_undegraded":"2023-09-29T21:42:44.878375+0000","last_fullsized":"2023-09-29T21:42:44.878375+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740750,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.223,"15min":0.217},"min":{"1min":0.159,"5min":0.151,"15min":0.151},"max":{"1min":0.24299999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.21299999999999999,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.25600000000000001,"5min":0.40500000000000003,"15min":0.442},"last":0.222}]},{"osd":1,"last update":"Fri Sep 29 21:42:09 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.23599999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.27800000000000002,"5min":0.372,"15min":1.044},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.25,"5min":0.26300000000000001,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.30199999999999999,"5min":0.39600000000000002,"15min":0.44900000000000001},"last":0.252}]}]},{"osd":0,"up_from":8,"seq":34359740749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.221,"15min":0.20300000000000001},"min":{"1min":0.157,"5min":0.153,"15min":0.13300000000000001},"max":{"1min":0.32200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.26100000000000001,"15min":0.26400000000000001},"min":{"1min":0.20200000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.29099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.25}]},{"osd":2,"last update":"Fri Sep 29 21:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.23200000000000001,"15min":0.22800000000000001},"min":{"1min":0.17100000000000001,"5min":0.157,"15min":0.157},"max":{"1min":0.30499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.253,"15min":0.249},"min":{"1min":0.19600000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.42099999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.23699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740747,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.13500000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.40100000000000002,"5min":0.435,"15min":0.435},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22,"15min":0.219},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.155},"max":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27000000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.313}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:42:49.710 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.711 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.711 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.711 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.712 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.712 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.712 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:42:49.712 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:42:49.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:49.910+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:50.525 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:50.524+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:50.922 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:50.921+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:51.556 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:51.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:51.882 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:51.881+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:52.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:52.509+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:52.885 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:52.884+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:53.487 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:53.486+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:53.908 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:53.907+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:54.502 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:54.502+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:54.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:54.892+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:55.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:55.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:55.923 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:55.923+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:56.529 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:56.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:56.934 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:56.934+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:57.562 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:57.561+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:57.889 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:57.888+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:58.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:58.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:58.856 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:58.855+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:59.552 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:42:59.551+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:42:59.842 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:42:59.842+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:00.528 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:00.527+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:00.817 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:00.817+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:01.490 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:01.490+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:01.787 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:01.786+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:02.467 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:02.466+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:02.792 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:02.791+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:03.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:03.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:03.835 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:03.834+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:04.457 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:04.456+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:04.810 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:04.809+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:05.465 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:05.463+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:05.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:05.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:06.509 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:06.508+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:06.829 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:06.828+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:07.543 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:07.543+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:07.846 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:07.845+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:08.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:08.558+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:08.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:08.872+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:09.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:09.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:09.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:43:09.902 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:09.901+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:43:10.036 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:43:10.047 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6103,"stamp":"2023-09-29T21:43:09.333929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001999"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3073,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:58.856201+0000","last_change":"2023-09-29T21:42:58.856201+0000","last_active":"2023-09-29T21:42:58.856201+0000","last_peered":"2023-09-29T21:42:58.856201+0000","last_clean":"2023-09-29T21:42:58.856201+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:42:58.856201+0000","last_undegraded":"2023-09-29T21:42:58.856201+0000","last_fullsized":"2023-09-29T21:42:58.856201+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12820,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:45.876320+0000","last_change":"2023-09-29T21:42:45.876320+0000","last_active":"2023-09-29T21:42:45.876320+0000","last_peered":"2023-09-29T21:42:45.876320+0000","last_clean":"2023-09-29T21:42:45.876320+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:42:45.876320+0000","last_undegraded":"2023-09-29T21:42:45.876320+0000","last_fullsized":"2023-09-29T21:42:45.876320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":374,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:48.957876+0000","last_change":"2023-09-29T21:42:48.957876+0000","last_active":"2023-09-29T21:42:48.957876+0000","last_peered":"2023-09-29T21:42:48.957876+0000","last_clean":"2023-09-29T21:42:48.957876+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:42:48.957876+0000","last_undegraded":"2023-09-29T21:42:48.957876+0000","last_fullsized":"2023-09-29T21:42:48.957876+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3558,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:52.885734+0000","last_change":"2023-09-29T21:42:52.885734+0000","last_active":"2023-09-29T21:42:52.885734+0000","last_peered":"2023-09-29T21:42:52.885734+0000","last_clean":"2023-09-29T21:42:52.885734+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:42:52.885734+0000","last_undegraded":"2023-09-29T21:42:52.885734+0000","last_fullsized":"2023-09-29T21:42:52.885734+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10899,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:42:57.889854+0000","last_change":"2023-09-29T21:42:57.889854+0000","last_active":"2023-09-29T21:42:57.889854+0000","last_peered":"2023-09-29T21:42:57.889854+0000","last_clean":"2023-09-29T21:42:57.889854+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:42:57.889854+0000","last_undegraded":"2023-09-29T21:42:57.889854+0000","last_fullsized":"2023-09-29T21:42:57.889854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12043,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:39:03.180330+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:39:03.180330+0000","last_peered":"2023-09-29T21:39:03.180330+0000","last_clean":"2023-09-29T21:39:03.180330+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:39:03.180330+0000","last_undegraded":"2023-09-29T21:39:03.180330+0000","last_fullsized":"2023-09-29T21:39:03.180330+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 392s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3254,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:02.792845+0000","last_change":"2023-09-29T21:43:02.792845+0000","last_active":"2023-09-29T21:43:02.792845+0000","last_peered":"2023-09-29T21:43:02.792845+0000","last_clean":"2023-09-29T21:43:02.792845+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:43:02.792845+0000","last_undegraded":"2023-09-29T21:43:02.792845+0000","last_fullsized":"2023-09-29T21:43:02.792845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740754,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.223,"15min":0.217},"min":{"1min":0.159,"5min":0.151,"15min":0.151},"max":{"1min":0.24299999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.183},{"interface":"front","average":{"1min":0.21299999999999999,"5min":0.23499999999999999,"15min":0.23599999999999999},"min":{"1min":0.17299999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.25600000000000001,"5min":0.40500000000000003,"15min":0.442},"last":0.20699999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:42:09 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.23599999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.27800000000000002,"5min":0.372,"15min":1.044},"last":0.223},{"interface":"front","average":{"1min":0.25,"5min":0.26300000000000001,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.30199999999999999,"5min":0.39600000000000002,"15min":0.44900000000000001},"last":0.24199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.20499999999999999},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.33800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.307,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26800000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.252,"15min":0.25},"min":{"1min":0.17799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35899999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.28699999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740751,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.13500000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.40100000000000002,"5min":0.435,"15min":0.435},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22,"15min":0.219},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.155},"max":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27000000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.27500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:43:10.049 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.050 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.050 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.050 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.050 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.051 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.051 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:10.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:43:10.415 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:43:10.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:43:10.590 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:10.589+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:43:10.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:43:10.913 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:10.913+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:43:11.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:43:11.481 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:43:11.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:43:11.618 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:11.617+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:11.862 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:43:11.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:43:11.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:11.898+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:12.243 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:43:12.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:43:12.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:12.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:12.626 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:43:12.637 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:43:12.931 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:12.930+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:13.636 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:13.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:13.979 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:13.979+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:14.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:14.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:14.958 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:14.957+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:15.654 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:15.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:15.955 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:15.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:16.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:16.676+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:16.981 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:16.981+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:17.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:17.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:17.999 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:17.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:18.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:18.659+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:18.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:18.986+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:19.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:19.637+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:20.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:20.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:20.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:20.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:21.016 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:21.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:21.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:21.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:22.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:22.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:22.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:22.669+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:23.030 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:23.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:23.648 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:23.647+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:24.015 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:24.015+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:24.630 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:24.629+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:25.000 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:24.999+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:25.609 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:25.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:25.996 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:25.995+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:26.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:26.610+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:27.038 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:27.037+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:27.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:27.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:28.088 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:28.087+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:28.610 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:28.609+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:29.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:29.130+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:29.589 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:29.588+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:30.169 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:30.169+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:30.578 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:30.578+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:31.187 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:31.186+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:31.542 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:31.541+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:32.235 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:32.234+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:32.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:32.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:32.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:43:32.995 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:43:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:43:33.010 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6114,"stamp":"2023-09-29T21:43:31.337221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001788"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3077,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:11.899354+0000","last_change":"2023-09-29T21:43:11.899354+0000","last_active":"2023-09-29T21:43:11.899354+0000","last_peered":"2023-09-29T21:43:11.899354+0000","last_clean":"2023-09-29T21:43:11.899354+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:43:11.899354+0000","last_undegraded":"2023-09-29T21:43:11.899354+0000","last_fullsized":"2023-09-29T21:43:11.899354+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12824,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:10.913892+0000","last_change":"2023-09-29T21:43:10.913892+0000","last_active":"2023-09-29T21:43:10.913892+0000","last_peered":"2023-09-29T21:43:10.913892+0000","last_clean":"2023-09-29T21:43:10.913892+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:43:10.913892+0000","last_undegraded":"2023-09-29T21:43:10.913892+0000","last_fullsized":"2023-09-29T21:43:10.913892+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":378,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:24.016124+0000","last_change":"2023-09-29T21:43:24.016124+0000","last_active":"2023-09-29T21:43:24.016124+0000","last_peered":"2023-09-29T21:43:24.016124+0000","last_clean":"2023-09-29T21:43:24.016124+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:43:24.016124+0000","last_undegraded":"2023-09-29T21:43:24.016124+0000","last_fullsized":"2023-09-29T21:43:24.016124+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3562,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:13.980137+0000","last_change":"2023-09-29T21:43:13.980137+0000","last_active":"2023-09-29T21:43:13.980137+0000","last_peered":"2023-09-29T21:43:13.980137+0000","last_clean":"2023-09-29T21:43:13.980137+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:43:13.980137+0000","last_undegraded":"2023-09-29T21:43:13.980137+0000","last_fullsized":"2023-09-29T21:43:13.980137+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10903,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:15.956280+0000","last_change":"2023-09-29T21:43:15.956280+0000","last_active":"2023-09-29T21:43:15.956280+0000","last_peered":"2023-09-29T21:43:15.956280+0000","last_clean":"2023-09-29T21:43:15.956280+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:43:15.956280+0000","last_undegraded":"2023-09-29T21:43:15.956280+0000","last_fullsized":"2023-09-29T21:43:15.956280+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3258,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:27.038891+0000","last_change":"2023-09-29T21:43:27.038891+0000","last_active":"2023-09-29T21:43:27.038891+0000","last_peered":"2023-09-29T21:43:27.038891+0000","last_clean":"2023-09-29T21:43:27.038891+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:43:27.038891+0000","last_undegraded":"2023-09-29T21:43:27.038891+0000","last_fullsized":"2023-09-29T21:43:27.038891+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740758,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":0.37,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.153,"15min":0.153},"max":{"1min":0.39600000000000002,"5min":0.40500000000000003,"15min":0.442},"last":0.19800000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.23899999999999999,"5min":0.23400000000000001,"15min":0.23899999999999999},"min":{"1min":0.184,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":1.044},"last":0.215},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25900000000000001,"15min":0.26300000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.23699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.20499999999999999},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.33800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.307,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.251}]},{"osd":2,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.252,"15min":0.25},"min":{"1min":0.17799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35899999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.27400000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740755,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20899999999999999,"15min":0.20499999999999999},"min":{"1min":0.13500000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.40100000000000002,"5min":0.435,"15min":0.435},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23699999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.23899999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22,"15min":0.219},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.155},"max":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27200000000000002,"15min":0.27000000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.27600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:43:33.011 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.011 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.012 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.012 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.012 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.012 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.012 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:33.013 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:43:33.241 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:33.240+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:33.586 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:33.585+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:34.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:34.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:34.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:34.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:35.185 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:35.184+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:35.567 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:35.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:36.190 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:36.189+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:36.606 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:36.605+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:37.195 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:37.195+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:37.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:37.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:38.238 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:38.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:38.559 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:38.559+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:39.199 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:39.198+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:39.568 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:39.567+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:40.180 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:40.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:40.546 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:40.546+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:41.179 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:41.179+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:41.533 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:41.532+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:42.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:42.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:42.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:42.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:43.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:43.255+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:43.581 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:43.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:44.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:44.244+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:44.620 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:44.620+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:45.263 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:45.262+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:45.608 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:45.608+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:46.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:46.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:46.611 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:46.610+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:47.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:47.226+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:47.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:47.648+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:48.278 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:48.276+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:48.644 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:48.644+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:49.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:49.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:49.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:49.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:50.227 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:50.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:50.651 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:50.651+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:51.215 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:51.215+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:51.645 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:51.645+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:52.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:52.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:52.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:52.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:53.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:43:53.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:53.223+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:53.368 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:43:53.371 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:43:53.383 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6125,"stamp":"2023-09-29T21:43:53.340360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001703"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3083,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:43.256622+0000","last_change":"2023-09-29T21:43:43.256622+0000","last_active":"2023-09-29T21:43:43.256622+0000","last_peered":"2023-09-29T21:43:43.256622+0000","last_clean":"2023-09-29T21:43:43.256622+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:43:43.256622+0000","last_undegraded":"2023-09-29T21:43:43.256622+0000","last_fullsized":"2023-09-29T21:43:43.256622+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12833,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:50.227564+0000","last_change":"2023-09-29T21:43:50.227564+0000","last_active":"2023-09-29T21:43:50.227564+0000","last_peered":"2023-09-29T21:43:50.227564+0000","last_clean":"2023-09-29T21:43:50.227564+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:43:50.227564+0000","last_undegraded":"2023-09-29T21:43:50.227564+0000","last_fullsized":"2023-09-29T21:43:50.227564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":384,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:48.276588+0000","last_change":"2023-09-29T21:43:48.276588+0000","last_active":"2023-09-29T21:43:48.276588+0000","last_peered":"2023-09-29T21:43:48.276588+0000","last_clean":"2023-09-29T21:43:48.276588+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:43:48.276588+0000","last_undegraded":"2023-09-29T21:43:48.276588+0000","last_fullsized":"2023-09-29T21:43:48.276588+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3568,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:45.263555+0000","last_change":"2023-09-29T21:43:45.263555+0000","last_active":"2023-09-29T21:43:45.263555+0000","last_peered":"2023-09-29T21:43:45.263555+0000","last_clean":"2023-09-29T21:43:45.263555+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:43:45.263555+0000","last_undegraded":"2023-09-29T21:43:45.263555+0000","last_fullsized":"2023-09-29T21:43:45.263555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10909,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:46.246582+0000","last_change":"2023-09-29T21:43:46.246582+0000","last_active":"2023-09-29T21:43:46.246582+0000","last_peered":"2023-09-29T21:43:46.246582+0000","last_clean":"2023-09-29T21:43:46.246582+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:43:46.246582+0000","last_undegraded":"2023-09-29T21:43:46.246582+0000","last_fullsized":"2023-09-29T21:43:46.246582+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3264,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:43:49.256718+0000","last_change":"2023-09-29T21:43:49.256718+0000","last_active":"2023-09-29T21:43:49.256718+0000","last_peered":"2023-09-29T21:43:49.256718+0000","last_clean":"2023-09-29T21:43:49.256718+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:43:49.256718+0000","last_undegraded":"2023-09-29T21:43:49.256718+0000","last_fullsized":"2023-09-29T21:43:49.256718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740763,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":0.37,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.16800000000000001},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.153,"15min":0.153},"max":{"1min":0.39600000000000002,"5min":0.40500000000000003,"15min":0.442},"last":0.20100000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.23899999999999999,"5min":0.23400000000000001,"15min":0.23899999999999999},"min":{"1min":0.184,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":1.044},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25900000000000001,"15min":0.26300000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.17799999999999999},"max":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.23400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.20499999999999999},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.13300000000000001},"max":{"1min":0.33800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.252,"15min":0.26200000000000001},"min":{"1min":0.19500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.307,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.27100000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.436},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.252,"15min":0.25},"min":{"1min":0.17799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.35899999999999999,"5min":0.48899999999999999,"15min":0.504},"last":0.44600000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740760,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.35499999999999998,"5min":0.40100000000000002,"15min":0.435},"last":0.245},{"interface":"front","average":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.39300000000000002,"5min":0.43099999999999999,"15min":0.47399999999999998},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.219,"15min":0.22},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.38100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.44400000000000001,"5min":0.498,"15min":0.498},"last":0.30599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:43:53.384 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.384 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.385 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.385 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.385 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.385 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.385 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:43:53.386 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:43:53.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:53.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:54.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:54.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:54.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:54.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:55.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:55.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:55.677 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:55.677+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:56.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:56.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:56.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:56.714+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:57.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:57.331+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:57.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:57.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:58.329 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:58.329+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:58.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:58.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:59.296 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:43:59.296+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:43:59.699 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:43:59.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:00.341 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:00.341+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:00.664 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:00.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:01.343 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:01.343+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:01.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:01.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:02.339 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:02.339+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:02.741 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:02.741+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:03.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:03.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:03.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:03.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:04.381 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:04.381+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:04.792 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:04.793+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:05.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:05.353+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:05.768 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:05.768+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:06.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:06.305+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:06.788 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:06.788+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:07.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:07.307+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:07.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:07.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:08.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:08.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:08.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:08.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:09.315 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:09.315+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:09.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:09.711+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:10.287 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:10.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:10.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:10.739+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:11.280 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:11.280+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:11.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:11.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:12.245 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:12.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:12.750 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:12.750+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:13.202 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:13.202+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:13.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:44:13.704 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:44:13.706 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:44:13.717 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6135,"stamp":"2023-09-29T21:44:13.343708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002063"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3089,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:02.339882+0000","last_change":"2023-09-29T21:44:02.339882+0000","last_active":"2023-09-29T21:44:02.339882+0000","last_peered":"2023-09-29T21:44:02.339882+0000","last_clean":"2023-09-29T21:44:02.339882+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:44:02.339882+0000","last_undegraded":"2023-09-29T21:44:02.339882+0000","last_fullsized":"2023-09-29T21:44:02.339882+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12839,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:12.245753+0000","last_change":"2023-09-29T21:44:12.245753+0000","last_active":"2023-09-29T21:44:12.245753+0000","last_peered":"2023-09-29T21:44:12.245753+0000","last_clean":"2023-09-29T21:44:12.245753+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:44:12.245753+0000","last_undegraded":"2023-09-29T21:44:12.245753+0000","last_fullsized":"2023-09-29T21:44:12.245753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":390,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:09.315604+0000","last_change":"2023-09-29T21:44:09.315604+0000","last_active":"2023-09-29T21:44:09.315604+0000","last_peered":"2023-09-29T21:44:09.315604+0000","last_clean":"2023-09-29T21:44:09.315604+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:44:09.315604+0000","last_undegraded":"2023-09-29T21:44:09.315604+0000","last_fullsized":"2023-09-29T21:44:09.315604+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3574,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:04.382155+0000","last_change":"2023-09-29T21:44:04.382155+0000","last_active":"2023-09-29T21:44:04.382155+0000","last_peered":"2023-09-29T21:44:04.382155+0000","last_clean":"2023-09-29T21:44:04.382155+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:44:04.382155+0000","last_undegraded":"2023-09-29T21:44:04.382155+0000","last_fullsized":"2023-09-29T21:44:04.382155+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10915,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:05.353644+0000","last_change":"2023-09-29T21:44:05.353644+0000","last_active":"2023-09-29T21:44:05.353644+0000","last_peered":"2023-09-29T21:44:05.353644+0000","last_clean":"2023-09-29T21:44:05.353644+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:44:05.353644+0000","last_undegraded":"2023-09-29T21:44:05.353644+0000","last_fullsized":"2023-09-29T21:44:05.353644+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3270,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:10.287787+0000","last_change":"2023-09-29T21:44:10.287787+0000","last_active":"2023-09-29T21:44:10.287787+0000","last_peered":"2023-09-29T21:44:10.287787+0000","last_clean":"2023-09-29T21:44:10.287787+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:44:10.287787+0000","last_undegraded":"2023-09-29T21:44:10.287787+0000","last_fullsized":"2023-09-29T21:44:10.287787+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740767,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.217,"15min":0.218},"min":{"1min":0.151,"5min":0.151,"15min":0.151},"max":{"1min":0.37,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.191},{"interface":"front","average":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.153,"15min":0.153},"max":{"1min":0.39600000000000002,"5min":0.40500000000000003,"15min":0.442},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23999999999999999},"min":{"1min":0.19700000000000001,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.318,"5min":0.40799999999999997,"15min":1.044},"last":0.253},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.34200000000000003,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.26800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.20999999999999999},"min":{"1min":0.154,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.26300000000000001},"min":{"1min":0.19400000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.436,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.253,"15min":0.252},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.504},"last":0.36499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740764,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.35499999999999998,"5min":0.40100000000000002,"15min":0.435},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.39300000000000002,"5min":0.43099999999999999,"15min":0.47399999999999998},"last":0.23100000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.219,"15min":0.22},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.38100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.44400000000000001,"5min":0.498,"15min":0.498},"last":0.27400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:44:13.723 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.723 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.723 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.723 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.723 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.724 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.724 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:13.724 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:44:13.785 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:13.785+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:14.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:14.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:14.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:14.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:15.203 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:15.203+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:15.774 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:15.774+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:16.164 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:16.164+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:16.730 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:16.731+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:17.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:17.208+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:17.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:17.728+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:18.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:18.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:18.719 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:18.719+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:19.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:19.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:19.714 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:19.715+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:20.222 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:20.222+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:20.695 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:20.695+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:21.213 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:21.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:21.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:21.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:22.178 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:22.178+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:22.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:22.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:23.226 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:23.227+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:23.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:23.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:24.208 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:24.209+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:24.708 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:24.708+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:25.212 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:25.213+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:25.666 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:25.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:26.253 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:26.253+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:26.701 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:26.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:27.229 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:27.229+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:27.707 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:27.707+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:28.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:28.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:28.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:28.743+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:29.206 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:29.206+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:29.702 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:29.703+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:30.168 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:30.168+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:30.682 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:30.682+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:31.210 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:31.211+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:31.727 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:31.727+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:32.218 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:32.219+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:32.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:32.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:33.201 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:33.201+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:33.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:44:33.763 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:33.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:34.082 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:44:34.084 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:44:34.095 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6145,"stamp":"2023-09-29T21:44:33.347158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002048"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3092,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:17.208420+0000","last_change":"2023-09-29T21:44:17.208420+0000","last_active":"2023-09-29T21:44:17.208420+0000","last_peered":"2023-09-29T21:44:17.208420+0000","last_clean":"2023-09-29T21:44:17.208420+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:44:17.208420+0000","last_undegraded":"2023-09-29T21:44:17.208420+0000","last_fullsized":"2023-09-29T21:44:17.208420+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12839,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:12.245753+0000","last_change":"2023-09-29T21:44:12.245753+0000","last_active":"2023-09-29T21:44:12.245753+0000","last_peered":"2023-09-29T21:44:12.245753+0000","last_clean":"2023-09-29T21:44:12.245753+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:44:12.245753+0000","last_undegraded":"2023-09-29T21:44:12.245753+0000","last_fullsized":"2023-09-29T21:44:12.245753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":393,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:28.216864+0000","last_change":"2023-09-29T21:44:28.216864+0000","last_active":"2023-09-29T21:44:28.216864+0000","last_peered":"2023-09-29T21:44:28.216864+0000","last_clean":"2023-09-29T21:44:28.216864+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:44:28.216864+0000","last_undegraded":"2023-09-29T21:44:28.216864+0000","last_fullsized":"2023-09-29T21:44:28.216864+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3577,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:19.214077+0000","last_change":"2023-09-29T21:44:19.214077+0000","last_active":"2023-09-29T21:44:19.214077+0000","last_peered":"2023-09-29T21:44:19.214077+0000","last_clean":"2023-09-29T21:44:19.214077+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:44:19.214077+0000","last_undegraded":"2023-09-29T21:44:19.214077+0000","last_fullsized":"2023-09-29T21:44:19.214077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10918,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:26.254086+0000","last_change":"2023-09-29T21:44:26.254086+0000","last_active":"2023-09-29T21:44:26.254086+0000","last_peered":"2023-09-29T21:44:26.254086+0000","last_clean":"2023-09-29T21:44:26.254086+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:44:26.254086+0000","last_undegraded":"2023-09-29T21:44:26.254086+0000","last_fullsized":"2023-09-29T21:44:26.254086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3273,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:29.206876+0000","last_change":"2023-09-29T21:44:29.206876+0000","last_active":"2023-09-29T21:44:29.206876+0000","last_peered":"2023-09-29T21:44:29.206876+0000","last_clean":"2023-09-29T21:44:29.206876+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:44:29.206876+0000","last_undegraded":"2023-09-29T21:44:29.206876+0000","last_fullsized":"2023-09-29T21:44:29.206876+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740771,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21299999999999999,"15min":0.218},"min":{"1min":0.16800000000000001,"5min":0.151,"15min":0.151},"max":{"1min":0.27400000000000002,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.23699999999999999},"min":{"1min":0.18099999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.29999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23999999999999999},"min":{"1min":0.19700000000000001,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.318,"5min":0.40799999999999997,"15min":1.044},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.34200000000000003,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.20999999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.20999999999999999},"min":{"1min":0.154,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.17100000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.26300000000000001},"min":{"1min":0.19400000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.23400000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.436,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.253,"15min":0.252},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.504},"last":0.218}]}]},{"osd":1,"up_from":8,"seq":34359740768,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.13900000000000001,"5min":0.13500000000000001,"15min":0.114},"max":{"1min":0.35499999999999998,"5min":0.40100000000000002,"15min":0.435},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23899999999999999},"min":{"1min":0.17100000000000001,"5min":0.17000000000000001,"15min":0.13600000000000001},"max":{"1min":0.39300000000000002,"5min":0.43099999999999999,"15min":0.47399999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Sep 29 21:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.219,"15min":0.22},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.38100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.44400000000000001,"5min":0.498,"15min":0.498},"last":0.27300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:44:34.097 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.097 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.097 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.098 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.098 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.098 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.098 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:34.098 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:44:34.234 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:34.235+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:34.726 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:34.726+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:35.224 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:35.225+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:35.767 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:35.767+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:36.258 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:36.258+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:36.723 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:36.723+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:37.285 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:37.285+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:37.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:37.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:38.312 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:38.312+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:38.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:38.701+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:39.294 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:39.295+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:39.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:39.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:40.260 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:40.259+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:40.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:40.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:41.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:41.264+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:41.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:41.699+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:42.307 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:42.306+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:42.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:42.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:43.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:43.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:43.724 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:43.724+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:44.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:44.287+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:44.759 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:44.758+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:45.305 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:45.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:45.764 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:45.763+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:46.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:46.320+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:46.810 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:46.810+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:47.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:47.332+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:47.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:47.812+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:48.288 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:48.288+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:48.850 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:48.849+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:49.311 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:49.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:49.833 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:49.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:50.334 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:50.334+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:50.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:50.846+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:51.384 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:51.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:51.882 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:51.881+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:52.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:52.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:52.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:52.876+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:53.331 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:53.330+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:53.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:53.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:54.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:44:54.303 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:54.302+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:54.419 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:44:54.421 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:44:54.433 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6155,"stamp":"2023-09-29T21:44:53.350319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001802"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3095,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:37.285510+0000","last_change":"2023-09-29T21:44:37.285510+0000","last_active":"2023-09-29T21:44:37.285510+0000","last_peered":"2023-09-29T21:44:37.285510+0000","last_clean":"2023-09-29T21:44:37.285510+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:44:37.285510+0000","last_undegraded":"2023-09-29T21:44:37.285510+0000","last_fullsized":"2023-09-29T21:44:37.285510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12842,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:33.201351+0000","last_change":"2023-09-29T21:44:33.201351+0000","last_active":"2023-09-29T21:44:33.201351+0000","last_peered":"2023-09-29T21:44:33.201351+0000","last_clean":"2023-09-29T21:44:33.201351+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:44:33.201351+0000","last_undegraded":"2023-09-29T21:44:33.201351+0000","last_fullsized":"2023-09-29T21:44:33.201351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":396,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:49.311444+0000","last_change":"2023-09-29T21:44:49.311444+0000","last_active":"2023-09-29T21:44:49.311444+0000","last_peered":"2023-09-29T21:44:49.311444+0000","last_clean":"2023-09-29T21:44:49.311444+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:44:49.311444+0000","last_undegraded":"2023-09-29T21:44:49.311444+0000","last_fullsized":"2023-09-29T21:44:49.311444+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3580,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:40.260910+0000","last_change":"2023-09-29T21:44:40.260910+0000","last_active":"2023-09-29T21:44:40.260910+0000","last_peered":"2023-09-29T21:44:40.260910+0000","last_clean":"2023-09-29T21:44:40.260910+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:44:40.260910+0000","last_undegraded":"2023-09-29T21:44:40.260910+0000","last_fullsized":"2023-09-29T21:44:40.260910+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10921,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:44.288762+0000","last_change":"2023-09-29T21:44:44.288762+0000","last_active":"2023-09-29T21:44:44.288762+0000","last_peered":"2023-09-29T21:44:44.288762+0000","last_clean":"2023-09-29T21:44:44.288762+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:44:44.288762+0000","last_undegraded":"2023-09-29T21:44:44.288762+0000","last_fullsized":"2023-09-29T21:44:44.288762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3273,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:29.206876+0000","last_change":"2023-09-29T21:44:29.206876+0000","last_active":"2023-09-29T21:44:29.206876+0000","last_peered":"2023-09-29T21:44:29.206876+0000","last_clean":"2023-09-29T21:44:29.206876+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:44:29.206876+0000","last_undegraded":"2023-09-29T21:44:29.206876+0000","last_fullsized":"2023-09-29T21:44:29.206876+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740775,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21299999999999999,"15min":0.218},"min":{"1min":0.16800000000000001,"5min":0.151,"15min":0.151},"max":{"1min":0.27400000000000002,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.16800000000000001},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.23699999999999999},"min":{"1min":0.18099999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.29999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.222}]},{"osd":1,"last update":"Fri Sep 29 21:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23999999999999999},"min":{"1min":0.19700000000000001,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.318,"5min":0.40799999999999997,"15min":1.044},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.34200000000000003,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.254}]}]},{"osd":0,"up_from":8,"seq":34359740774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.20999999999999999},"min":{"1min":0.154,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.249,"15min":0.26300000000000001},"min":{"1min":0.19400000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.19600000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22900000000000001},"min":{"1min":0.17799999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.436,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.214},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.253,"15min":0.252},"min":{"1min":0.19,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.504},"last":0.25900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740772,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.13300000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.184},{"interface":"front","average":{"1min":0.248,"5min":0.24299999999999999,"15min":0.23999999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.222}]},{"osd":2,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.223,"15min":0.221},"min":{"1min":0.17799999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.34300000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27200000000000002},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.40200000000000002,"5min":0.498,"15min":0.498},"last":0.27200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:44:54.435 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.435 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.435 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.435 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.436 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.436 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.436 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:44:54.436 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:44:54.845 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:54.845+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:55.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:55.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:55.876 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:55.875+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:56.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:56.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:56.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:56.870+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:57.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:57.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:57.921 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:57.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:58.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:58.300+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:58.920 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:58.920+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:59.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:44:59.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:44:59.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:44:59.918+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:00.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:00.282+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:00.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:00.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:01.261 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:01.260+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:01.993 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:01.992+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:02.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:02.292+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:02.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:02.952+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:03.283 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:03.283+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:03.970 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:03.970+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:04.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:04.245+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:04.949 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:04.949+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:05.205 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:05.204+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:05.982 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:05.981+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:06.246 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:06.246+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:06.986 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:06.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:07.291 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:07.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:07.959 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:07.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:08.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:08.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:08.968 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:08.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:09.292 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:09.291+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:09.996 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:09.995+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:10.318 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:10.317+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:10.947 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:10.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:11.328 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:11.327+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:11.974 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:11.974+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:12.362 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:12.361+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:12.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:12.999+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:13.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:13.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:13.994 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:13.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:14.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:14.392+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:14.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:45:14.816 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:45:14.819 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:45:14.830 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6165,"stamp":"2023-09-29T21:45:13.353121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001634"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3098,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:06.247255+0000","last_change":"2023-09-29T21:45:06.247255+0000","last_active":"2023-09-29T21:45:06.247255+0000","last_peered":"2023-09-29T21:45:06.247255+0000","last_clean":"2023-09-29T21:45:06.247255+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:45:06.247255+0000","last_undegraded":"2023-09-29T21:45:06.247255+0000","last_fullsized":"2023-09-29T21:45:06.247255+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12845,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:04.246558+0000","last_change":"2023-09-29T21:45:04.246558+0000","last_active":"2023-09-29T21:45:04.246558+0000","last_peered":"2023-09-29T21:45:04.246558+0000","last_clean":"2023-09-29T21:45:04.246558+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:45:04.246558+0000","last_undegraded":"2023-09-29T21:45:04.246558+0000","last_fullsized":"2023-09-29T21:45:04.246558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":396,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:49.311444+0000","last_change":"2023-09-29T21:44:49.311444+0000","last_active":"2023-09-29T21:44:49.311444+0000","last_peered":"2023-09-29T21:44:49.311444+0000","last_clean":"2023-09-29T21:44:49.311444+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:44:49.311444+0000","last_undegraded":"2023-09-29T21:44:49.311444+0000","last_fullsized":"2023-09-29T21:44:49.311444+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3583,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:09.292550+0000","last_change":"2023-09-29T21:45:09.292550+0000","last_active":"2023-09-29T21:45:09.292550+0000","last_peered":"2023-09-29T21:45:09.292550+0000","last_clean":"2023-09-29T21:45:09.292550+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:45:09.292550+0000","last_undegraded":"2023-09-29T21:45:09.292550+0000","last_fullsized":"2023-09-29T21:45:09.292550+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10921,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:44:44.288762+0000","last_change":"2023-09-29T21:44:44.288762+0000","last_active":"2023-09-29T21:44:44.288762+0000","last_peered":"2023-09-29T21:44:44.288762+0000","last_clean":"2023-09-29T21:44:44.288762+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:44:44.288762+0000","last_undegraded":"2023-09-29T21:44:44.288762+0000","last_fullsized":"2023-09-29T21:44:44.288762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12044,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:41:07.642626+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:41:07.642626+0000","last_peered":"2023-09-29T21:41:07.642626+0000","last_clean":"2023-09-29T21:41:07.642626+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:41:07.642626+0000","last_undegraded":"2023-09-29T21:41:07.642626+0000","last_fullsized":"2023-09-29T21:41:07.642626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 516s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3276,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:01.261795+0000","last_change":"2023-09-29T21:45:01.261795+0000","last_active":"2023-09-29T21:45:01.261795+0000","last_peered":"2023-09-29T21:45:01.261795+0000","last_clean":"2023-09-29T21:45:01.261795+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:45:01.261795+0000","last_undegraded":"2023-09-29T21:45:01.261795+0000","last_fullsized":"2023-09-29T21:45:01.261795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740779,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21299999999999999,"15min":0.218},"min":{"1min":0.16800000000000001,"5min":0.151,"15min":0.151},"max":{"1min":0.27400000000000002,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.23699999999999999},"min":{"1min":0.18099999999999999,"5min":0.153,"15min":0.153},"max":{"1min":0.29999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.19800000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23999999999999999},"min":{"1min":0.19700000000000001,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.318,"5min":0.40799999999999997,"15min":1.044},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.34200000000000003,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.23300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23300000000000001,"15min":0.21099999999999999},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.17499999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.39200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.253},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.186,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.44600000000000001,"15min":0.504},"last":0.222}]}]},{"osd":1,"up_from":8,"seq":34359740776,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.13300000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.24299999999999999,"15min":0.23999999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.223,"15min":0.221},"min":{"1min":0.17799999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.34300000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27200000000000002},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.40200000000000002,"5min":0.498,"15min":0.498},"last":0.32800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:45:14.832 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.832 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.833 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.833 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.833 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.833 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.833 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:14.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:45:14.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:14.984+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:45:15.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:45:15.353 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:15.352+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:15.525 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:45:15.537 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:45:15.862 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:45:15.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:45:16.009 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:16.008+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:45:16.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:45:16.392 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:16.391+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:16.518 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:45:16.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:45:16.843 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:45:16.853 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:45:16.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:16.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:17.179 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:45:17.191 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:45:17.365 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:17.364+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:18.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:18.035+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:18.376 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:18.376+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:19.015 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:19.014+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:19.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:19.396+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:20.059 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:20.059+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:20.401 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:20.400+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:21.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:21.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:21.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:21.410+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:22.081 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:22.080+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:22.399 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:22.399+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:23.036 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:23.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:23.416 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:23.415+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:24.079 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:24.078+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:24.402 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:24.401+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:25.035 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:25.034+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:25.429 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:25.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:25.991 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:25.990+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:26.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:26.414+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:26.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:26.944+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:27.393 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:27.393+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:27.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:27.958+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:28.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:28.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:28.955 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:28.954+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:29.405 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:29.404+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:29.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:29.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:30.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:30.356+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:31.020 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:31.020+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:31.345 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:31.344+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:32.070 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:32.070+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:32.357 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:32.356+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:33.050 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:33.049+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:33.387 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:33.386+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:34.073 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:34.073+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:34.414 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:34.413+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:35.116 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:35.115+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:35.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:35.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:36.151 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:36.150+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:36.383 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:36.383+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:37.138 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:37.137+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:37.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:45:37.349 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:37.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:38.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:38.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:38.315 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:45:38.318 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:45:38.332 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6177,"stamp":"2023-09-29T21:45:37.356521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001649"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3102,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:27.394092+0000","last_change":"2023-09-29T21:45:27.394092+0000","last_active":"2023-09-29T21:45:27.394092+0000","last_peered":"2023-09-29T21:45:27.394092+0000","last_clean":"2023-09-29T21:45:27.394092+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:45:27.394092+0000","last_undegraded":"2023-09-29T21:45:27.394092+0000","last_fullsized":"2023-09-29T21:45:27.394092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12849,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:30.358020+0000","last_change":"2023-09-29T21:45:30.358020+0000","last_active":"2023-09-29T21:45:30.358020+0000","last_peered":"2023-09-29T21:45:30.358020+0000","last_clean":"2023-09-29T21:45:30.358020+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:45:30.358020+0000","last_undegraded":"2023-09-29T21:45:30.358020+0000","last_fullsized":"2023-09-29T21:45:30.358020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":400,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:15.353465+0000","last_change":"2023-09-29T21:45:15.353429+0000","last_active":"2023-09-29T21:45:15.353465+0000","last_peered":"2023-09-29T21:45:15.353465+0000","last_clean":"2023-09-29T21:45:15.353465+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:45:15.353465+0000","last_undegraded":"2023-09-29T21:45:15.353465+0000","last_fullsized":"2023-09-29T21:45:15.353465+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3587,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:32.357811+0000","last_change":"2023-09-29T21:45:32.357811+0000","last_active":"2023-09-29T21:45:32.357811+0000","last_peered":"2023-09-29T21:45:32.357811+0000","last_clean":"2023-09-29T21:45:32.357811+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:45:32.357811+0000","last_undegraded":"2023-09-29T21:45:32.357811+0000","last_fullsized":"2023-09-29T21:45:32.357811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10925,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:14.393710+0000","last_change":"2023-09-29T21:45:14.393690+0000","last_active":"2023-09-29T21:45:14.393710+0000","last_peered":"2023-09-29T21:45:14.393710+0000","last_clean":"2023-09-29T21:45:14.393710+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:45:14.393710+0000","last_undegraded":"2023-09-29T21:45:14.393710+0000","last_fullsized":"2023-09-29T21:45:14.393710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3280,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:24.402345+0000","last_change":"2023-09-29T21:45:24.402345+0000","last_active":"2023-09-29T21:45:24.402345+0000","last_peered":"2023-09-29T21:45:24.402345+0000","last_clean":"2023-09-29T21:45:24.402345+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:45:24.402345+0000","last_undegraded":"2023-09-29T21:45:24.402345+0000","last_fullsized":"2023-09-29T21:45:24.402345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740783,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21099999999999999,"15min":0.218},"min":{"1min":0.161,"5min":0.151,"15min":0.151},"max":{"1min":0.32900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22700000000000001,"15min":0.23799999999999999},"min":{"1min":0.185,"5min":0.153,"15min":0.153},"max":{"1min":0.35999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":1.044},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.34699999999999998,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.27500000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23300000000000001,"15min":0.21099999999999999},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.17499999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.221}]},{"osd":2,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.39200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.186,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.44600000000000001,"15min":0.504},"last":0.20200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740780,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.13300000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.24299999999999999,"15min":0.23999999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.186}]},{"osd":2,"last update":"Fri Sep 29 21:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.223,"15min":0.221},"min":{"1min":0.17799999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.34300000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27200000000000002},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.40200000000000002,"5min":0.498,"15min":0.498},"last":0.26000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:45:38.334 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.334 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.334 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.334 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.335 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.335 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.335 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:38.335 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:45:38.371 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:38.370+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:39.051 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:39.051+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:39.411 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:39.410+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:40.005 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:40.004+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:40.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:40.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:41.043 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:41.042+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:41.487 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:41.486+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:41.997 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:41.997+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:42.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:42.481+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:43.041 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:43.040+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:43.499 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:43.500+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:43.998 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:43.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:44.522 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:44.523+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:44.953 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:44.953+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:45.488 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:45.488+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:45.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:45.906+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:46.524 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:46.525+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:46.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:46.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:47.544 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:47.545+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:47.913 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:47.913+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:48.510 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:48.511+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:48.907 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:48.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:49.541 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:49.541+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:49.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:49.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:50.640 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:50.551+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:50.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:50.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:51.515 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:51.515+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:51.942 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:51.942+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:52.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:52.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:52.922 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:52.923+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:53.530 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:53.531+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:53.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:53.967+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:54.517 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:54.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:54.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:54.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:55.496 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:55.496+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:55.901 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:55.901+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:56.489 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:56.489+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:56.918 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:56.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:57.535 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:57.535+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:57.879 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:57.879+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:58.337 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:45:58.549 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:58.550+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:58.692 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:45:58.694 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:45:58.706 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6187,"stamp":"2023-09-29T21:45:57.359715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001724"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3105,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:41.487542+0000","last_change":"2023-09-29T21:45:41.487542+0000","last_active":"2023-09-29T21:45:41.487542+0000","last_peered":"2023-09-29T21:45:41.487542+0000","last_clean":"2023-09-29T21:45:41.487542+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:45:41.487542+0000","last_undegraded":"2023-09-29T21:45:41.487542+0000","last_fullsized":"2023-09-29T21:45:41.487542+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12852,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:47.545105+0000","last_change":"2023-09-29T21:45:47.545105+0000","last_active":"2023-09-29T21:45:47.545105+0000","last_peered":"2023-09-29T21:45:47.545105+0000","last_clean":"2023-09-29T21:45:47.545105+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:45:47.545105+0000","last_undegraded":"2023-09-29T21:45:47.545105+0000","last_fullsized":"2023-09-29T21:45:47.545105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":406,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:51.515478+0000","last_change":"2023-09-29T21:45:51.515478+0000","last_active":"2023-09-29T21:45:51.515478+0000","last_peered":"2023-09-29T21:45:51.515478+0000","last_clean":"2023-09-29T21:45:51.515478+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:45:51.515478+0000","last_undegraded":"2023-09-29T21:45:51.515478+0000","last_fullsized":"2023-09-29T21:45:51.515478+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3590,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:48.511226+0000","last_change":"2023-09-29T21:45:48.511226+0000","last_active":"2023-09-29T21:45:48.511226+0000","last_peered":"2023-09-29T21:45:48.511226+0000","last_clean":"2023-09-29T21:45:48.511226+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:45:48.511226+0000","last_undegraded":"2023-09-29T21:45:48.511226+0000","last_fullsized":"2023-09-29T21:45:48.511226+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10931,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:49.541765+0000","last_change":"2023-09-29T21:45:49.541765+0000","last_active":"2023-09-29T21:45:49.541765+0000","last_peered":"2023-09-29T21:45:49.541765+0000","last_clean":"2023-09-29T21:45:49.541765+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:45:49.541765+0000","last_undegraded":"2023-09-29T21:45:49.541765+0000","last_fullsized":"2023-09-29T21:45:49.541765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3286,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:52.489869+0000","last_change":"2023-09-29T21:45:52.489869+0000","last_active":"2023-09-29T21:45:52.489869+0000","last_peered":"2023-09-29T21:45:52.489869+0000","last_clean":"2023-09-29T21:45:52.489869+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:45:52.489869+0000","last_undegraded":"2023-09-29T21:45:52.489869+0000","last_fullsized":"2023-09-29T21:45:52.489869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740787,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21099999999999999,"15min":0.218},"min":{"1min":0.161,"5min":0.151,"15min":0.151},"max":{"1min":0.32900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22700000000000001,"15min":0.23799999999999999},"min":{"1min":0.185,"5min":0.153,"15min":0.153},"max":{"1min":0.35999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.21199999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":1.044},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.34699999999999998,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.25800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23300000000000001,"15min":0.21099999999999999},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.251,"5min":0.248,"15min":0.26200000000000001},"min":{"1min":0.17499999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.40899999999999997,"5min":0.42699999999999999,"15min":0.45300000000000001},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22900000000000001},"min":{"1min":0.17299999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.39200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.185},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.254,"15min":0.254},"min":{"1min":0.186,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.36499999999999999,"5min":0.44600000000000001,"15min":0.504},"last":0.24099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740784,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20899999999999999},"min":{"1min":0.16200000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.378,"5min":0.41599999999999998,"15min":0.435},"last":0.13500000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.246,"15min":0.24199999999999999},"min":{"1min":0.186,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.41599999999999998,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.16900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22500000000000001,"15min":0.222},"min":{"1min":0.16600000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.40200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.45200000000000001,"5min":0.498,"15min":0.498},"last":0.22900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:45:58.708 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.708 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.708 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.709 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.709 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.709 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.709 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:45:58.709 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:45:58.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:58.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:59.556 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:45:59.557+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:45:59.817 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:45:59.818+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:00.532 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:00.533+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:00.834 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:00.834+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:01.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:01.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:01.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:01.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:02.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:02.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:02.842 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:02.843+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:03.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:03.439+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:03.852 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:03.852+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:04.437 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:04.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:04.853 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:04.853+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:05.397 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:05.397+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:05.848 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:05.848+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:06.424 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:06.424+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:06.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:06.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:07.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:07.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:07.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:07.886+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:08.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:08.453+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:08.841 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:08.841+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:09.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:09.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:09.880 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:09.880+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:10.449 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:10.449+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:10.891 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:10.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:11.432 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:11.433+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:11.886 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:11.887+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:12.440 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:12.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:12.846 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:12.847+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:13.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:13.462+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:13.804 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:13.804+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:14.500 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:14.500+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:14.813 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:14.813+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:15.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:15.540+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:15.765 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:15.765+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:16.586 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:16.587+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:16.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:16.743+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:17.619 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:17.619+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:17.789 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:17.790+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:18.658 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:18.658+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:18.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:46:18.743 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:18.743+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:19.028 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:46:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:46:19.043 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6197,"stamp":"2023-09-29T21:46:17.362923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001837"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3108,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:45:53.531227+0000","last_change":"2023-09-29T21:45:53.531227+0000","last_active":"2023-09-29T21:45:53.531227+0000","last_peered":"2023-09-29T21:45:53.531227+0000","last_clean":"2023-09-29T21:45:53.531227+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:45:53.531227+0000","last_undegraded":"2023-09-29T21:45:53.531227+0000","last_fullsized":"2023-09-29T21:45:53.531227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12855,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:00.532982+0000","last_change":"2023-09-29T21:46:00.532982+0000","last_active":"2023-09-29T21:46:00.532982+0000","last_peered":"2023-09-29T21:46:00.532982+0000","last_clean":"2023-09-29T21:46:00.532982+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:46:00.532982+0000","last_undegraded":"2023-09-29T21:46:00.532982+0000","last_fullsized":"2023-09-29T21:46:00.532982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":409,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:05.397661+0000","last_change":"2023-09-29T21:46:05.397661+0000","last_active":"2023-09-29T21:46:05.397661+0000","last_peered":"2023-09-29T21:46:05.397661+0000","last_clean":"2023-09-29T21:46:05.397661+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:46:05.397661+0000","last_undegraded":"2023-09-29T21:46:05.397661+0000","last_fullsized":"2023-09-29T21:46:05.397661+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3593,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:03.439834+0000","last_change":"2023-09-29T21:46:03.439834+0000","last_active":"2023-09-29T21:46:03.439834+0000","last_peered":"2023-09-29T21:46:03.439834+0000","last_clean":"2023-09-29T21:46:03.439834+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:46:03.439834+0000","last_undegraded":"2023-09-29T21:46:03.439834+0000","last_fullsized":"2023-09-29T21:46:03.439834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10934,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:04.437464+0000","last_change":"2023-09-29T21:46:04.437464+0000","last_active":"2023-09-29T21:46:04.437464+0000","last_peered":"2023-09-29T21:46:04.437464+0000","last_clean":"2023-09-29T21:46:04.437464+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:46:04.437464+0000","last_undegraded":"2023-09-29T21:46:04.437464+0000","last_fullsized":"2023-09-29T21:46:04.437464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3289,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:10.449351+0000","last_change":"2023-09-29T21:46:10.449351+0000","last_active":"2023-09-29T21:46:10.449351+0000","last_peered":"2023-09-29T21:46:10.449351+0000","last_clean":"2023-09-29T21:46:10.449351+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:46:10.449351+0000","last_undegraded":"2023-09-29T21:46:10.449351+0000","last_fullsized":"2023-09-29T21:46:10.449351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740791,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21099999999999999,"15min":0.218},"min":{"1min":0.161,"5min":0.151,"15min":0.151},"max":{"1min":0.32900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.23499999999999999,"5min":0.22700000000000001,"15min":0.23799999999999999},"min":{"1min":0.185,"5min":0.153,"15min":0.153},"max":{"1min":0.35999999999999999,"5min":0.39600000000000002,"15min":0.442},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Sep 29 21:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.189,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":1.044},"last":0.371},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.26400000000000001},"min":{"1min":0.20999999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.34699999999999998,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.39800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23499999999999999,"15min":0.215},"min":{"1min":0.155,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.38600000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.188},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.253,"15min":0.26300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.45300000000000001},"last":0.23499999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.436,"15min":0.44700000000000001},"last":0.248},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.254},"min":{"1min":0.19500000000000001,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.41999999999999998,"5min":0.44600000000000001,"15min":0.504},"last":0.223}]}]},{"osd":1,"up_from":8,"seq":34359740788,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20899999999999999},"min":{"1min":0.16200000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.378,"5min":0.41599999999999998,"15min":0.435},"last":0.187},{"interface":"front","average":{"1min":0.255,"5min":0.246,"15min":0.24199999999999999},"min":{"1min":0.186,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.41599999999999998,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.22}]},{"osd":2,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22500000000000001,"15min":0.222},"min":{"1min":0.16600000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.40200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.45200000000000001,"5min":0.498,"15min":0.498},"last":0.24299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:46:19.044 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.045 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.045 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.045 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.045 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.046 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.046 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:19.046 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:46:19.657 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:19.657+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:19.705 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:19.705+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:20.696 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:20.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:20.706 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:20.707+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:21.684 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:21.685+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:21.691 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:21.691+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:22.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:22.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:22.711 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:22.712+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:23.642 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:23.642+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:23.674 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:23.675+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:24.653 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:24.653+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:24.707 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:24.708+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:25.634 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:25.635+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:25.697 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:25.697+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:26.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:26.655+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:26.744 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:26.744+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:27.665 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:27.665+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:27.756 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:27.757+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:28.674 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:28.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:28.722 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:28.723+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:29.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:29.667+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:29.737 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:29.737+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:30.649 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:30.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:30.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:30.759+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:31.632 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:31.632+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:31.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:31.771+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:32.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:32.656+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:32.786 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:32.787+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:33.638 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:33.638+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:33.819 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:33.819+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:34.639 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:34.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:34.780 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:34.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:35.628 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:35.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:35.781 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:35.780+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:36.640 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:36.639+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:36.759 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:36.758+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:37.679 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:37.678+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:37.732 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:37.731+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:38.655 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:38.654+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:38.748 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:38.747+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:39.047 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:46:39.369 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:46:39.371 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:46:39.383 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6208,"stamp":"2023-09-29T21:46:39.366416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002054"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3111,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:15.540424+0000","last_change":"2023-09-29T21:46:15.540424+0000","last_active":"2023-09-29T21:46:15.540424+0000","last_peered":"2023-09-29T21:46:15.540424+0000","last_clean":"2023-09-29T21:46:15.540424+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:46:15.540424+0000","last_undegraded":"2023-09-29T21:46:15.540424+0000","last_fullsized":"2023-09-29T21:46:15.540424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12858,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:18.658698+0000","last_change":"2023-09-29T21:46:18.658698+0000","last_active":"2023-09-29T21:46:18.658698+0000","last_peered":"2023-09-29T21:46:18.658698+0000","last_clean":"2023-09-29T21:46:18.658698+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:46:18.658698+0000","last_undegraded":"2023-09-29T21:46:18.658698+0000","last_fullsized":"2023-09-29T21:46:18.658698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":409,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:05.397661+0000","last_change":"2023-09-29T21:46:05.397661+0000","last_active":"2023-09-29T21:46:05.397661+0000","last_peered":"2023-09-29T21:46:05.397661+0000","last_clean":"2023-09-29T21:46:05.397661+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:46:05.397661+0000","last_undegraded":"2023-09-29T21:46:05.397661+0000","last_fullsized":"2023-09-29T21:46:05.397661+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3596,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:27.757471+0000","last_change":"2023-09-29T21:46:27.757471+0000","last_active":"2023-09-29T21:46:27.757471+0000","last_peered":"2023-09-29T21:46:27.757471+0000","last_clean":"2023-09-29T21:46:27.757471+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:46:27.757471+0000","last_undegraded":"2023-09-29T21:46:27.757471+0000","last_fullsized":"2023-09-29T21:46:27.757471+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10937,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:30.759809+0000","last_change":"2023-09-29T21:46:30.759809+0000","last_active":"2023-09-29T21:46:30.759809+0000","last_peered":"2023-09-29T21:46:30.759809+0000","last_clean":"2023-09-29T21:46:30.759809+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:46:30.759809+0000","last_undegraded":"2023-09-29T21:46:30.759809+0000","last_fullsized":"2023-09-29T21:46:30.759809+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3289,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:10.449351+0000","last_change":"2023-09-29T21:46:10.449351+0000","last_active":"2023-09-29T21:46:10.449351+0000","last_peered":"2023-09-29T21:46:10.449351+0000","last_clean":"2023-09-29T21:46:10.449351+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:46:10.449351+0000","last_undegraded":"2023-09-29T21:46:10.449351+0000","last_fullsized":"2023-09-29T21:46:10.449351+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740796,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20899999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.151,"15min":0.151},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":1.0600000000000001},"last":0.192},{"interface":"front","average":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23899999999999999},"min":{"1min":0.19800000000000001,"5min":0.16800000000000001,"15min":0.153},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.224}]},{"osd":1,"last update":"Fri Sep 29 21:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.245},"min":{"1min":0.184,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.434,"5min":0.44700000000000001,"15min":1.044},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26700000000000002},"min":{"1min":0.23300000000000001,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.41999999999999998,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.26600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23499999999999999,"15min":0.215},"min":{"1min":0.155,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.38600000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.253,"15min":0.26300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.45300000000000001},"last":0.25700000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.436,"15min":0.44700000000000001},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.254},"min":{"1min":0.19500000000000001,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.41999999999999998,"5min":0.44600000000000001,"15min":0.504},"last":0.44600000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740793,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.20999999999999999,"15min":0.20899999999999999},"min":{"1min":0.16200000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.378,"5min":0.41599999999999998,"15min":0.435},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.255,"5min":0.246,"15min":0.24199999999999999},"min":{"1min":0.186,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.41599999999999998,"5min":0.45100000000000001,"15min":0.47399999999999998},"last":0.23599999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.22500000000000001,"15min":0.222},"min":{"1min":0.16600000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.40200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27400000000000002},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":0.45200000000000001,"5min":0.498,"15min":0.498},"last":0.27100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:46:39.385 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.385 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.385 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.386 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.386 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.386 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.386 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:39.386 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:46:39.697 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:39.697+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:39.761 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:39.760+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:40.675 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:40.674+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:40.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:40.724+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:41.669 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:41.668+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:41.774 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:41.773+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:42.667 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:42.666+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:42.753 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:42.752+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:43.694 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:43.694+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:43.763 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:43.761+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:44.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:44.692+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:44.771 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:44.770+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:45.703 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:45.702+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:45.778 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:45.778+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:46.739 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:46.738+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:46.800 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:46.799+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:47.748 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:47.748+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:47.826 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:47.826+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:48.773 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:48.771+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:48.853 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:48.852+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:49.782 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:49.781+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:49.898 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:49.897+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:50.826 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:50.825+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:50.862 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:50.862+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:51.803 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:51.802+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:51.864 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:51.864+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:52.807 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:52.806+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:52.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:52.865+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:53.821 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:53.820+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:53.866 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:53.863+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:54.823 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:54.822+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:54.912 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:54.911+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:55.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:55.830+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:55.954 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:55.954+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:56.827 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:56.827+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:56.942 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:56.939+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:57.795 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:57.794+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:57.895 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:57.895+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:58.815 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:58.814+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:58.858 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:58.857+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:59.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:46:59.707 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:46:59.710 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:46:59.722 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6218,"stamp":"2023-09-29T21:46:59.369346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001796"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3114,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:42.753950+0000","last_change":"2023-09-29T21:46:42.753950+0000","last_active":"2023-09-29T21:46:42.753950+0000","last_peered":"2023-09-29T21:46:42.753950+0000","last_clean":"2023-09-29T21:46:42.753950+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:46:42.753950+0000","last_undegraded":"2023-09-29T21:46:42.753950+0000","last_fullsized":"2023-09-29T21:46:42.753950+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:44.771433+0000","last_change":"2023-09-29T21:46:44.771433+0000","last_active":"2023-09-29T21:46:44.771433+0000","last_peered":"2023-09-29T21:46:44.771433+0000","last_clean":"2023-09-29T21:46:44.771433+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:46:44.771433+0000","last_undegraded":"2023-09-29T21:46:44.771433+0000","last_fullsized":"2023-09-29T21:46:44.771433+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":415,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:53.864546+0000","last_change":"2023-09-29T21:46:53.864546+0000","last_active":"2023-09-29T21:46:53.864546+0000","last_peered":"2023-09-29T21:46:53.864546+0000","last_clean":"2023-09-29T21:46:53.864546+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:46:53.864546+0000","last_undegraded":"2023-09-29T21:46:53.864546+0000","last_fullsized":"2023-09-29T21:46:53.864546+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3599,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:48.853312+0000","last_change":"2023-09-29T21:46:48.853312+0000","last_active":"2023-09-29T21:46:48.853312+0000","last_peered":"2023-09-29T21:46:48.853312+0000","last_clean":"2023-09-29T21:46:48.853312+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:46:48.853312+0000","last_undegraded":"2023-09-29T21:46:48.853312+0000","last_fullsized":"2023-09-29T21:46:48.853312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10940,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:52.866797+0000","last_change":"2023-09-29T21:46:52.866797+0000","last_active":"2023-09-29T21:46:52.866797+0000","last_peered":"2023-09-29T21:46:52.866797+0000","last_clean":"2023-09-29T21:46:52.866797+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:46:52.866797+0000","last_undegraded":"2023-09-29T21:46:52.866797+0000","last_fullsized":"2023-09-29T21:46:52.866797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3295,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:46:56.940721+0000","last_change":"2023-09-29T21:46:56.940721+0000","last_active":"2023-09-29T21:46:56.940721+0000","last_peered":"2023-09-29T21:46:56.940721+0000","last_clean":"2023-09-29T21:46:56.940721+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:46:56.940721+0000","last_undegraded":"2023-09-29T21:46:56.940721+0000","last_fullsized":"2023-09-29T21:46:56.940721+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740800,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20899999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.151,"15min":0.151},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":1.0600000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23899999999999999},"min":{"1min":0.19800000000000001,"5min":0.16800000000000001,"15min":0.153},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.245},"min":{"1min":0.184,"5min":0.17899999999999999,"15min":0.17699999999999999},"max":{"1min":0.434,"5min":0.44700000000000001,"15min":1.044},"last":0.311},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.26400000000000001,"15min":0.26700000000000002},"min":{"1min":0.23300000000000001,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.41999999999999998,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.29299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23499999999999999,"15min":0.215},"min":{"1min":0.155,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.38600000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.253,"15min":0.26300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.41399999999999998,"5min":0.42199999999999999,"15min":0.45300000000000001},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Sep 29 21:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.23100000000000001},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.39600000000000002,"5min":0.436,"15min":0.44700000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.254},"min":{"1min":0.19500000000000001,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.41999999999999998,"5min":0.44600000000000001,"15min":0.504},"last":0.441}]}]},{"osd":1,"up_from":8,"seq":34359740797,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"min":{"1min":0.13500000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.247,"15min":0.24199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.28100000000000003}]},{"osd":2,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.22900000000000001,"15min":0.224},"min":{"1min":0.17000000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.436,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.255},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.28100000000000003,"15min":0.27600000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.30399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:46:59.723 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.723 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.724 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.724 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.724 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.724 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.724 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:46:59.725 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:46:59.859 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:46:59.858+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:46:59.907 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:46:59.906+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:00.869 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:00.868+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:00.903 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:00.902+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:01.870 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:01.869+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:01.874 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:01.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:02.872 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:02.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:02.872 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:02.871+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:03.875 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:03.874+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:03.917 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:03.916+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:04.892 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:04.891+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:04.956 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:04.955+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:05.900 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:05.899+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:05.992 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:05.991+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:06.932 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:06.931+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:07.025 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:07.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:07.904 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:07.903+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:08.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:08.055+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:08.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:08.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:09.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:09.099+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:09.908 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:09.907+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:10.057 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:10.056+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:10.906 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:10.905+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:11.039 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:11.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:11.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:11.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:12.044 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:12.043+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:12.969 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:12.968+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:13.022 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:13.021+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:13.999 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:13.998+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:14.031 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:14.030+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:14.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:14.971+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:15.062 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:15.061+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:15.987 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:15.986+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:16.047 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:16.046+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:16.972 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:16.972+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:17.046 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:17.045+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:17.940 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:17.939+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:18.079 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:18.078+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:18.903 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:18.902+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:19.058 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:19.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:19.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:47:19.930 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:19.929+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:20.040 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:20.039+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:20.075 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:47:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:47:20.088 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6228,"stamp":"2023-09-29T21:47:19.372817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002013"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3120,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:14.031739+0000","last_change":"2023-09-29T21:47:14.031739+0000","last_active":"2023-09-29T21:47:14.031739+0000","last_peered":"2023-09-29T21:47:14.031739+0000","last_clean":"2023-09-29T21:47:14.031739+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:47:14.031739+0000","last_undegraded":"2023-09-29T21:47:14.031739+0000","last_fullsized":"2023-09-29T21:47:14.031739+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12867,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:15.063079+0000","last_change":"2023-09-29T21:47:15.063079+0000","last_active":"2023-09-29T21:47:15.063079+0000","last_peered":"2023-09-29T21:47:15.063079+0000","last_clean":"2023-09-29T21:47:15.063079+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:47:15.063079+0000","last_undegraded":"2023-09-29T21:47:15.063079+0000","last_fullsized":"2023-09-29T21:47:15.063079+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":418,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:10.057915+0000","last_change":"2023-09-29T21:47:10.057915+0000","last_active":"2023-09-29T21:47:10.057915+0000","last_peered":"2023-09-29T21:47:10.057915+0000","last_clean":"2023-09-29T21:47:10.057915+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:47:10.057915+0000","last_undegraded":"2023-09-29T21:47:10.057915+0000","last_fullsized":"2023-09-29T21:47:10.057915+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3605,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:16.047164+0000","last_change":"2023-09-29T21:47:16.047164+0000","last_active":"2023-09-29T21:47:16.047164+0000","last_peered":"2023-09-29T21:47:16.047164+0000","last_clean":"2023-09-29T21:47:16.047164+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:47:16.047164+0000","last_undegraded":"2023-09-29T21:47:16.047164+0000","last_fullsized":"2023-09-29T21:47:16.047164+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10943,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:08.056207+0000","last_change":"2023-09-29T21:47:08.056207+0000","last_active":"2023-09-29T21:47:08.056207+0000","last_peered":"2023-09-29T21:47:08.056207+0000","last_clean":"2023-09-29T21:47:08.056207+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:47:08.056207+0000","last_undegraded":"2023-09-29T21:47:08.056207+0000","last_fullsized":"2023-09-29T21:47:08.056207+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12045,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:43:12.241446+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:43:12.241446+0000","last_peered":"2023-09-29T21:43:12.241446+0000","last_clean":"2023-09-29T21:43:12.241446+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:43:12.241446+0000","last_undegraded":"2023-09-29T21:43:12.241446+0000","last_fullsized":"2023-09-29T21:43:12.241446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 641s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3298,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:11.040359+0000","last_change":"2023-09-29T21:47:11.040359+0000","last_active":"2023-09-29T21:47:11.040359+0000","last_peered":"2023-09-29T21:47:11.040359+0000","last_clean":"2023-09-29T21:47:11.040359+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:47:11.040359+0000","last_undegraded":"2023-09-29T21:47:11.040359+0000","last_fullsized":"2023-09-29T21:47:11.040359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740805,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20899999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.151,"15min":0.151},"max":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":1.0600000000000001},"last":0.16200000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23899999999999999},"min":{"1min":0.19800000000000001,"5min":0.16800000000000001,"15min":0.153},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.246,"15min":0.246},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.317,"5min":0.44700000000000001,"15min":1.044},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.21199999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.36499999999999999,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.191}]}]},{"osd":0,"up_from":8,"seq":34359740803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.218},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.26300000000000001},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.224}]},{"osd":2,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.25900000000000001,"15min":0.25600000000000001},"min":{"1min":0.189,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.504},"last":0.20899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740801,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"min":{"1min":0.13500000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.247,"15min":0.24199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.22900000000000001,"15min":0.224},"min":{"1min":0.17000000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.436,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.214},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.28100000000000003,"15min":0.27600000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.27600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:47:20.090 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.090 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.091 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.091 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.091 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.091 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.091 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:20.092 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-09-29T21:47:20.438 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-09-29T21:47:20.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-09-29T21:47:20.813 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-09-29T21:47:20.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-09-29T21:47:20.926 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:20.925+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:21.005 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:21.004+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:21.143 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-09-29T21:47:21.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-09-29T21:47:21.488 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-09-29T21:47:21.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-09-29T21:47:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-09-29T21:47:21.839 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-09-29T21:47:21.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:21.946+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:21.983 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:21.982+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:22.196 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-09-29T21:47:22.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-09-29T21:47:22.541 INFO:teuthology.orchestra.run.smithi143.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-09-29T21:47:22.553 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:47:22.946 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:22.945+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:22.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:22.998+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:23.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:23.928+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:24.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:24.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:24.898 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:24.898+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:24.998 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:24.997+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:25.931 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:25.930+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:26.026 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:26.025+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:26.964 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:26.964+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:27.011 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:27.010+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:27.975 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:27.974+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:27.989 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:27.988+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:28.962 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:28.961+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:29.037 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:29.036+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:29.936 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:29.935+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:30.018 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:30.018+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:30.933 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:30.932+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:31.056 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:31.056+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:31.949 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:31.948+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:32.054 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:32.053+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:32.986 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:32.985+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:33.033 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:33.032+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:34.020 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:34.019+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:34.055 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:34.054+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:35.027 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:35.026+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:35.090 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:35.089+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:36.001 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:36.000+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:36.097 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:36.096+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:37.024 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:37.023+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:37.086 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:37.086+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:38.039 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:38.039+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:38.059 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:38.058+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:39.048 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:39.048+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:39.076 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:39.076+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:40.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:40.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:40.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:40.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:40.992 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:40.993+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:41.161 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:41.161+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:42.013 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:42.013+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:42.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:42.115+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:42.554 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:47:42.931 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:47:42.934 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:47:42.946 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6239,"stamp":"2023-09-29T21:47:41.376388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001887"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:35.027531+0000","last_change":"2023-09-29T21:47:35.027531+0000","last_active":"2023-09-29T21:47:35.027531+0000","last_peered":"2023-09-29T21:47:35.027531+0000","last_clean":"2023-09-29T21:47:35.027531+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:47:35.027531+0000","last_undegraded":"2023-09-29T21:47:35.027531+0000","last_fullsized":"2023-09-29T21:47:35.027531+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12871,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:31.950058+0000","last_change":"2023-09-29T21:47:31.950058+0000","last_active":"2023-09-29T21:47:31.950058+0000","last_peered":"2023-09-29T21:47:31.950058+0000","last_clean":"2023-09-29T21:47:31.950058+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:47:31.950058+0000","last_undegraded":"2023-09-29T21:47:31.950058+0000","last_fullsized":"2023-09-29T21:47:31.950058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":422,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:26.026571+0000","last_change":"2023-09-29T21:47:26.026571+0000","last_active":"2023-09-29T21:47:26.026571+0000","last_peered":"2023-09-29T21:47:26.026571+0000","last_clean":"2023-09-29T21:47:26.026571+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:47:26.026571+0000","last_undegraded":"2023-09-29T21:47:26.026571+0000","last_fullsized":"2023-09-29T21:47:26.026571+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3606,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:16.047190+0000","last_change":"2023-09-29T21:47:16.047164+0000","last_active":"2023-09-29T21:47:16.047190+0000","last_peered":"2023-09-29T21:47:16.047190+0000","last_clean":"2023-09-29T21:47:16.047190+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:47:16.047190+0000","last_undegraded":"2023-09-29T21:47:16.047190+0000","last_fullsized":"2023-09-29T21:47:16.047190+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10947,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:21.983626+0000","last_change":"2023-09-29T21:47:21.983626+0000","last_active":"2023-09-29T21:47:21.983626+0000","last_peered":"2023-09-29T21:47:21.983626+0000","last_clean":"2023-09-29T21:47:21.983626+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:47:21.983626+0000","last_undegraded":"2023-09-29T21:47:21.983626+0000","last_fullsized":"2023-09-29T21:47:21.983626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12046,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:45:16.840893+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:45:16.840893+0000","last_peered":"2023-09-29T21:45:16.840893+0000","last_clean":"2023-09-29T21:45:16.840893+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:45:16.840893+0000","last_undegraded":"2023-09-29T21:45:16.840893+0000","last_fullsized":"2023-09-29T21:45:16.840893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 766s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3299,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:11.040377+0000","last_change":"2023-09-29T21:47:11.040359+0000","last_active":"2023-09-29T21:47:11.040377+0000","last_peered":"2023-09-29T21:47:11.040377+0000","last_clean":"2023-09-29T21:47:11.040377+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:47:11.040377+0000","last_undegraded":"2023-09-29T21:47:11.040377+0000","last_fullsized":"2023-09-29T21:47:11.040377+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740809,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21099999999999999,"15min":0.22},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.28999999999999998,"5min":0.41999999999999998,"15min":1.0600000000000001},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23999999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.153},"max":{"1min":0.32800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.22900000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.246,"15min":0.246},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.317,"5min":0.44700000000000001,"15min":1.044},"last":0.219},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.21199999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.36499999999999999,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.26500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.218},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.245},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.26300000000000001},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.25900000000000001,"15min":0.25600000000000001},"min":{"1min":0.189,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.504},"last":0.38200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740805,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"min":{"1min":0.13500000000000001,"5min":0.13300000000000001,"15min":0.114},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.435},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.247,"15min":0.24199999999999999},"min":{"1min":0.16900000000000001,"5min":0.16300000000000001,"15min":0.13600000000000001},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Sep 29 21:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.22600000000000001,"5min":0.22900000000000001,"15min":0.224},"min":{"1min":0.17000000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.436,"5min":0.44400000000000001,"15min":0.44400000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.28100000000000003,"15min":0.27600000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:47:42.947 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.947 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.948 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.948 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.948 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.948 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.948 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:47:42.949 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:47:43.024 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:43.025+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:43.144 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:43.145+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:44.006 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:44.006+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:44.098 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:44.098+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:44.977 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:44.977+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:45.056 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:45.057+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:45.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:45.950+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:46.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:46.091+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:46.985 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:46.985+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:47.101 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:47.102+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:47.950 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:47.950+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:48.110 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:48.111+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:48.962 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:48.962+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:49.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:49.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:49.966 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:49.966+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:50.175 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:50.175+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:50.945 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:50.945+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:51.131 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:51.131+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:51.937 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:51.937+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:52.100 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:52.100+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:52.919 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:52.919+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:53.066 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:53.067+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:53.963 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:53.963+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:54.106 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:54.106+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:54.983 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:54.983+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:55.152 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:55.153+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:55.952 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:55.952+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:56.165 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:56.165+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:56.914 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:56.914+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:57.124 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:57.125+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:57.868 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:57.869+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:58.172 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:58.172+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:58.871 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:58.871+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:59.207 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:47:59.207+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:47:59.831 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:47:59.831+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:00.232 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:00.232+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:00.832 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:00.832+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:01.216 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:01.216+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:01.793 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:01.793+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:02.256 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:02.256+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:02.756 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:02.756+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:02.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:48:03.268 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:48:03.269 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:03.269+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:48:03.281 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6249,"stamp":"2023-09-29T21:48:01.379469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:35.027531+0000","last_change":"2023-09-29T21:47:35.027531+0000","last_active":"2023-09-29T21:47:35.027531+0000","last_peered":"2023-09-29T21:47:35.027531+0000","last_clean":"2023-09-29T21:47:35.027531+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:47:35.027531+0000","last_undegraded":"2023-09-29T21:47:35.027531+0000","last_fullsized":"2023-09-29T21:47:35.027531+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12874,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:53.067178+0000","last_change":"2023-09-29T21:47:53.067178+0000","last_active":"2023-09-29T21:47:53.067178+0000","last_peered":"2023-09-29T21:47:53.067178+0000","last_clean":"2023-09-29T21:47:53.067178+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:47:53.067178+0000","last_undegraded":"2023-09-29T21:47:53.067178+0000","last_fullsized":"2023-09-29T21:47:53.067178+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":425,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:49.132154+0000","last_change":"2023-09-29T21:47:49.132154+0000","last_active":"2023-09-29T21:47:49.132154+0000","last_peered":"2023-09-29T21:47:49.132154+0000","last_clean":"2023-09-29T21:47:49.132154+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:47:49.132154+0000","last_undegraded":"2023-09-29T21:47:49.132154+0000","last_fullsized":"2023-09-29T21:47:49.132154+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3609,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:44.098745+0000","last_change":"2023-09-29T21:47:44.098745+0000","last_active":"2023-09-29T21:47:44.098745+0000","last_peered":"2023-09-29T21:47:44.098745+0000","last_clean":"2023-09-29T21:47:44.098745+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:47:44.098745+0000","last_undegraded":"2023-09-29T21:47:44.098745+0000","last_fullsized":"2023-09-29T21:47:44.098745+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10950,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:46.091944+0000","last_change":"2023-09-29T21:47:46.091944+0000","last_active":"2023-09-29T21:47:46.091944+0000","last_peered":"2023-09-29T21:47:46.091944+0000","last_clean":"2023-09-29T21:47:46.091944+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:47:46.091944+0000","last_undegraded":"2023-09-29T21:47:46.091944+0000","last_fullsized":"2023-09-29T21:47:46.091944+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12046,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:45:16.840893+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:45:16.840893+0000","last_peered":"2023-09-29T21:45:16.840893+0000","last_clean":"2023-09-29T21:45:16.840893+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:45:16.840893+0000","last_undegraded":"2023-09-29T21:45:16.840893+0000","last_fullsized":"2023-09-29T21:45:16.840893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 766s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3302,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:47:41.161707+0000","last_change":"2023-09-29T21:47:41.161707+0000","last_active":"2023-09-29T21:47:41.161707+0000","last_peered":"2023-09-29T21:47:41.161707+0000","last_clean":"2023-09-29T21:47:41.161707+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:47:41.161707+0000","last_undegraded":"2023-09-29T21:47:41.161707+0000","last_fullsized":"2023-09-29T21:47:41.161707+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740813,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21099999999999999,"15min":0.22},"min":{"1min":0.14899999999999999,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.28999999999999998,"5min":0.41999999999999998,"15min":1.0600000000000001},"last":0.188},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23999999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.153},"max":{"1min":0.32800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.23599999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.246,"15min":0.246},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.317,"5min":0.44700000000000001,"15min":1.044},"last":0.215},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.26800000000000002},"min":{"1min":0.21199999999999999,"5min":0.19600000000000001,"15min":0.188},"max":{"1min":0.36499999999999999,"5min":0.42599999999999999,"15min":0.44900000000000001},"last":0.28399999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23599999999999999,"15min":0.218},"min":{"1min":0.14999999999999999,"5min":0.14599999999999999,"15min":0.14399999999999999},"max":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.251},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.26300000000000001},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23300000000000001},"min":{"1min":0.17499999999999999,"5min":0.16900000000000001,"15min":0.157},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.25900000000000001,"15min":0.25600000000000001},"min":{"1min":0.189,"5min":0.17799999999999999,"15min":0.17399999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.504},"last":0.28799999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740809,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"min":{"1min":0.156,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.435},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.18099999999999999,"5min":0.16300000000000001,"15min":0.16200000000000001},"max":{"1min":0.41799999999999998,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.22500000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.22600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.41899999999999998,"5min":0.436,"15min":0.44400000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27700000000000002},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46600000000000003,"5min":0.47599999999999998,"15min":0.498},"last":0.20599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:48:03.283 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.283 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.283 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.283 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.283 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.284 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.284 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:03.284 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:48:03.744 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:03.744+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:04.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:04.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:04.770 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:04.770+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:05.265 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:05.265+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:05.754 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:05.755+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:06.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:06.293+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:06.801 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:06.801+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:07.254 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:07.254+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:07.800 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:07.800+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:08.236 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:08.237+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:08.798 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:08.799+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:09.286 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:09.286+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:09.781 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:09.782+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:10.321 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:10.321+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:10.757 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:10.757+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:11.309 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:11.309+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:11.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:11.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:12.274 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:12.274+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:12.711 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:12.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:13.300 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:13.301+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:13.753 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:13.754+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:14.333 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:14.333+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:14.733 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:14.733+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:15.290 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:15.290+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:15.684 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:15.684+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:16.304 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:16.304+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:16.650 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:16.650+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:17.293 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:17.294+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:17.633 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:17.633+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:18.314 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:18.314+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:18.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:18.660+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:19.310 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:19.310+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:19.689 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:19.689+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:20.348 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:20.348+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:20.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:20.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:21.370 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:21.371+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:21.700 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:21.700+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:22.386 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:22.387+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:22.745 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:22.745+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:23.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:48:23.438 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:23.437+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:23.646 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:48:23.647 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:48:23.660 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6260,"stamp":"2023-09-29T21:48:23.382941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001748"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3130,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:11.309921+0000","last_change":"2023-09-29T21:48:11.309921+0000","last_active":"2023-09-29T21:48:11.309921+0000","last_peered":"2023-09-29T21:48:11.309921+0000","last_clean":"2023-09-29T21:48:11.309921+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:48:11.309921+0000","last_undegraded":"2023-09-29T21:48:11.309921+0000","last_fullsized":"2023-09-29T21:48:11.309921+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12877,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:10.321473+0000","last_change":"2023-09-29T21:48:10.321473+0000","last_active":"2023-09-29T21:48:10.321473+0000","last_peered":"2023-09-29T21:48:10.321473+0000","last_clean":"2023-09-29T21:48:10.321473+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:48:10.321473+0000","last_undegraded":"2023-09-29T21:48:10.321473+0000","last_fullsized":"2023-09-29T21:48:10.321473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":431,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:22.387111+0000","last_change":"2023-09-29T21:48:22.387111+0000","last_active":"2023-09-29T21:48:22.387111+0000","last_peered":"2023-09-29T21:48:22.387111+0000","last_clean":"2023-09-29T21:48:22.387111+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:48:22.387111+0000","last_undegraded":"2023-09-29T21:48:22.387111+0000","last_fullsized":"2023-09-29T21:48:22.387111+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3615,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:17.294200+0000","last_change":"2023-09-29T21:48:17.294200+0000","last_active":"2023-09-29T21:48:17.294200+0000","last_peered":"2023-09-29T21:48:17.294200+0000","last_clean":"2023-09-29T21:48:17.294200+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:48:17.294200+0000","last_undegraded":"2023-09-29T21:48:17.294200+0000","last_fullsized":"2023-09-29T21:48:17.294200+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10956,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:21.371000+0000","last_change":"2023-09-29T21:48:21.371000+0000","last_active":"2023-09-29T21:48:21.371000+0000","last_peered":"2023-09-29T21:48:21.371000+0000","last_clean":"2023-09-29T21:48:21.371000+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:48:21.371000+0000","last_undegraded":"2023-09-29T21:48:21.371000+0000","last_fullsized":"2023-09-29T21:48:21.371000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12046,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:45:16.840893+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:45:16.840893+0000","last_peered":"2023-09-29T21:45:16.840893+0000","last_clean":"2023-09-29T21:45:16.840893+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:45:16.840893+0000","last_undegraded":"2023-09-29T21:45:16.840893+0000","last_fullsized":"2023-09-29T21:45:16.840893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 766s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3308,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:16.304668+0000","last_change":"2023-09-29T21:48:16.304668+0000","last_active":"2023-09-29T21:48:16.304668+0000","last_peered":"2023-09-29T21:48:16.304668+0000","last_clean":"2023-09-29T21:48:16.304668+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:48:16.304668+0000","last_undegraded":"2023-09-29T21:48:16.304668+0000","last_fullsized":"2023-09-29T21:48:16.304668+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.215,"15min":0.221},"min":{"1min":0.16900000000000001,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.0600000000000001},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24199999999999999,"15min":0.24099999999999999},"min":{"1min":0.21299999999999999,"5min":0.17899999999999999,"15min":0.153},"max":{"1min":0.437,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.19500000000000001}]},{"osd":1,"last update":"Fri Sep 29 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.19700000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.74099999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.26900000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.188},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.23899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.26400000000000001,"5min":0.24299999999999999,"15min":0.223},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14399999999999999},"max":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.186,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23400000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.157},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.25800000000000001},"min":{"1min":0.187,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.504},"last":0.40999999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740814,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"min":{"1min":0.156,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.435},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.18099999999999999,"5min":0.16300000000000001,"15min":0.16200000000000001},"max":{"1min":0.41799999999999998,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.251}]},{"osd":2,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.22600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.41899999999999998,"5min":0.436,"15min":0.44400000000000001},"last":0.309},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27700000000000002},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46600000000000003,"5min":0.47599999999999998,"15min":0.498},"last":0.224}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:48:23.663 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.663 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.663 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.664 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.664 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.664 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.665 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:23.665 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:48:23.776 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:23.776+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:24.388 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:24.388+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:24.778 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:24.778+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:25.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:25.418+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:25.731 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:25.732+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:26.447 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:26.447+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:26.712 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:26.712+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:27.472 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:27.473+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:27.693 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:27.693+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:28.471 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:28.471+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:28.716 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:28.716+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:29.513 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:29.513+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:29.698 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:29.698+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:30.492 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:30.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:30.660 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:30.661+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:31.469 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:31.469+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:31.626 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:31.627+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:32.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:32.461+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:32.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:32.577+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:33.442 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:33.440+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:33.569 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:33.568+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:34.465 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:34.464+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:34.538 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:34.537+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:35.425 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:35.425+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:35.510 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:35.510+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:36.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:36.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:36.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:36.555+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:37.453 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:37.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:37.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:37.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:38.500 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:38.500+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:38.580 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:38.579+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:39.467 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:39.466+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:39.560 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:39.560+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:40.473 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:40.472+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:40.548 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:40.547+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:41.457 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:41.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:41.584 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:41.583+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:42.417 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:42.416+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:42.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:42.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:43.455 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:43.454+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:43.582 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:43.581+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:43.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:48:44.006 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:48:44.008 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:48:44.019 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6270,"stamp":"2023-09-29T21:48:43.386222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001871"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3136,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:36.467302+0000","last_change":"2023-09-29T21:48:36.467302+0000","last_active":"2023-09-29T21:48:36.467302+0000","last_peered":"2023-09-29T21:48:36.467302+0000","last_clean":"2023-09-29T21:48:36.467302+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:48:36.467302+0000","last_undegraded":"2023-09-29T21:48:36.467302+0000","last_fullsized":"2023-09-29T21:48:36.467302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12883,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:34.465635+0000","last_change":"2023-09-29T21:48:34.465635+0000","last_active":"2023-09-29T21:48:34.465635+0000","last_peered":"2023-09-29T21:48:34.465635+0000","last_clean":"2023-09-29T21:48:34.465635+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:48:34.465635+0000","last_undegraded":"2023-09-29T21:48:34.465635+0000","last_fullsized":"2023-09-29T21:48:34.465635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":434,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:32.462799+0000","last_change":"2023-09-29T21:48:32.462799+0000","last_active":"2023-09-29T21:48:32.462799+0000","last_peered":"2023-09-29T21:48:32.462799+0000","last_clean":"2023-09-29T21:48:32.462799+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:48:32.462799+0000","last_undegraded":"2023-09-29T21:48:32.462799+0000","last_fullsized":"2023-09-29T21:48:32.462799+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3621,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:41.458259+0000","last_change":"2023-09-29T21:48:41.458259+0000","last_active":"2023-09-29T21:48:41.458259+0000","last_peered":"2023-09-29T21:48:41.458259+0000","last_clean":"2023-09-29T21:48:41.458259+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:48:41.458259+0000","last_undegraded":"2023-09-29T21:48:41.458259+0000","last_fullsized":"2023-09-29T21:48:41.458259+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10962,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:42.417443+0000","last_change":"2023-09-29T21:48:42.417443+0000","last_active":"2023-09-29T21:48:42.417443+0000","last_peered":"2023-09-29T21:48:42.417443+0000","last_clean":"2023-09-29T21:48:42.417443+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:48:42.417443+0000","last_undegraded":"2023-09-29T21:48:42.417443+0000","last_fullsized":"2023-09-29T21:48:42.417443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12046,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:45:16.840893+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:45:16.840893+0000","last_peered":"2023-09-29T21:45:16.840893+0000","last_clean":"2023-09-29T21:45:16.840893+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:45:16.840893+0000","last_undegraded":"2023-09-29T21:45:16.840893+0000","last_fullsized":"2023-09-29T21:45:16.840893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 766s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3314,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:40.473316+0000","last_change":"2023-09-29T21:48:40.473316+0000","last_active":"2023-09-29T21:48:40.473316+0000","last_peered":"2023-09-29T21:48:40.473316+0000","last_clean":"2023-09-29T21:48:40.473316+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:48:40.473316+0000","last_undegraded":"2023-09-29T21:48:40.473316+0000","last_fullsized":"2023-09-29T21:48:40.473316+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740822,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.215,"15min":0.221},"min":{"1min":0.16900000000000001,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.0600000000000001},"last":0.186},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24199999999999999,"15min":0.24099999999999999},"min":{"1min":0.21299999999999999,"5min":0.17899999999999999,"15min":0.153},"max":{"1min":0.437,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.20799999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.19700000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.74099999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.26900000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.188},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.245}]}]},{"osd":0,"up_from":8,"seq":34359740820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.26400000000000001,"5min":0.24299999999999999,"15min":0.223},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14399999999999999},"max":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.186,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23400000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.157},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.25800000000000001},"min":{"1min":0.187,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.504},"last":0.49299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740818,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.20799999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"min":{"1min":0.156,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.435},"last":0.16200000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.18099999999999999,"5min":0.16300000000000001,"15min":0.16200000000000001},"max":{"1min":0.41799999999999998,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.20100000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.22600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":0.41899999999999998,"5min":0.436,"15min":0.44400000000000001},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27700000000000002},"min":{"1min":0.215,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46600000000000003,"5min":0.47599999999999998,"15min":0.498},"last":0.23300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:48:44.020 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.021 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.021 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.021 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.021 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.022 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.022 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:48:44.022 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-29T21:48:44.428 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:44.428+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:44.577 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:44.576+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:45.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:45.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:45.532 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:45.531+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:46.452 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:46.452+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:46.524 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:46.523+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:47.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:47.430+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:47.488 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:47.487+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:48.439 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:48.438+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:48.473 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:48.472+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:49.431 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:49.431+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:49.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:49.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:50.443 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:50.442+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:50.530 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:50.529+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:51.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:51.458+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:51.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:51.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:52.458 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:52.457+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:52.588 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:52.587+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:53.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:53.478+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:53.551 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:53.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:54.478 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:54.477+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:54.506 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:54.506+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:55.474 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:55.474+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:55.502 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:55.501+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:56.434 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:56.433+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:56.518 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:56.517+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:57.456 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:57.455+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:57.568 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:57.568+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:58.500 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:58.499+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:58.554 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:58.553+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:59.540 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:48:59.539+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:48:59.550 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:48:59.550+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:00.493 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:49:00.492+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:00.565 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:49:00.565+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:01.466 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:49:01.465+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:01.545 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:49:01.544+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:02.421 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:49:02.421+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:02.557 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:49:02.557+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:03.412 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:49:03.411+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:03.517 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T21:49:03.516+0000 7f305c966700 -1 osd.0 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1382 3.3 3:ddc2ae2a:::rbd_data.29767bf0fbb3.00000000000000e4:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:04.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T21:49:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T21:49:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T21:49:04.402 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":6280,"stamp":"2023-09-29T21:49:03.389274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1874605599,"num_objects":698,"num_object_clones":2,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":147,"num_read":68254,"num_read_kb":491494,"num_write":27769,"num_write_kb":8902418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35202,"ondisk_log_size":35202,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":789068,"kb_used_data":592820,"kb_used_omap":0,"kb_used_meta":196224,"kb_avail":282326452,"statfs":{"total":289910292480,"available":289102286848,"internally_reserved":0,"allocated":607047680,"data_stored":604734375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":200933376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001764"},"pg_stats":[{"pgid":"2.7","version":"162'1997","reported_seq":3032,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:31.000795+0000","last_change":"2023-09-29T21:34:31.000771+0000","last_active":"2023-09-29T21:34:31.000795+0000","last_peered":"2023-09-29T21:34:31.000795+0000","last_clean":"2023-09-29T21:34:31.000795+0000","last_became_active":"2023-09-29T18:25:04.585028+0000","last_became_peered":"2023-09-29T18:25:04.585028+0000","last_unstale":"2023-09-29T21:34:31.000795+0000","last_undegraded":"2023-09-29T21:34:31.000795+0000","last_fullsized":"2023-09-29T21:34:31.000795+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1997","last_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_deep_scrub":"162'1997","last_deep_scrub_stamp":"2023-09-29T21:34:31.000681+0000","last_clean_scrub_stamp":"2023-09-29T21:34:31.000681+0000","objects_scrubbed":53,"log_size":1997,"ondisk_log_size":1997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:15:51.787184+0000","scrub_duration":0.21323355699999999,"objects_trimmed":0,"snaptrim_duration":0.000310279,"stat_sum":{"num_bytes":202349219,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":846,"num_read_kb":11404,"num_write":1830,"num_write_kb":1031989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"159'2017","reported_seq":3139,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:55.502731+0000","last_change":"2023-09-29T21:48:55.502731+0000","last_active":"2023-09-29T21:48:55.502731+0000","last_peered":"2023-09-29T21:48:55.502731+0000","last_clean":"2023-09-29T21:48:55.502731+0000","last_became_active":"2023-09-29T18:25:04.584993+0000","last_became_peered":"2023-09-29T18:25:04.584993+0000","last_unstale":"2023-09-29T21:48:55.502731+0000","last_undegraded":"2023-09-29T21:48:55.502731+0000","last_fullsized":"2023-09-29T21:48:55.502731+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2017,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000237431,"stat_sum":{"num_bytes":255713444,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":741,"num_read_kb":7918,"num_write":1864,"num_write_kb":1187124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"162'1990","reported_seq":2927,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.002902+0000","last_change":"2023-09-29T21:34:34.002879+0000","last_active":"2023-09-29T21:34:34.002902+0000","last_peered":"2023-09-29T21:34:34.002902+0000","last_clean":"2023-09-29T21:34:34.002902+0000","last_became_active":"2023-09-29T18:25:04.585204+0000","last_became_peered":"2023-09-29T18:25:04.585204+0000","last_unstale":"2023-09-29T21:34:34.002902+0000","last_undegraded":"2023-09-29T21:34:34.002902+0000","last_fullsized":"2023-09-29T21:34:34.002902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1990","last_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_deep_scrub":"162'1990","last_deep_scrub_stamp":"2023-09-29T21:34:34.002801+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.002801+0000","objects_scrubbed":55,"log_size":1990,"ondisk_log_size":1990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:42.650482+0000","scrub_duration":0.20602388699999999,"objects_trimmed":0,"snaptrim_duration":0.00040511300000000002,"stat_sum":{"num_bytes":189886564,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":691,"num_read_kb":8245,"num_write":1794,"num_write_kb":1073451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":33,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"162'4380","reported_seq":9124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:32.806559+0000","last_change":"2023-09-29T21:34:32.806524+0000","last_active":"2023-09-29T21:34:32.806559+0000","last_peered":"2023-09-29T21:34:32.806559+0000","last_clean":"2023-09-29T21:34:32.806559+0000","last_became_active":"2023-09-29T18:25:11.715361+0000","last_became_peered":"2023-09-29T18:25:11.715361+0000","last_unstale":"2023-09-29T21:34:32.806559+0000","last_undegraded":"2023-09-29T21:34:32.806559+0000","last_fullsized":"2023-09-29T21:34:32.806559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'4380","last_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_deep_scrub":"162'4380","last_deep_scrub_stamp":"2023-09-29T21:34:32.806423+0000","last_clean_scrub_stamp":"2023-09-29T21:34:32.806423+0000","objects_scrubbed":51,"log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:29:19.752063+0000","scrub_duration":0.17606071300000001,"objects_trimmed":0,"snaptrim_duration":0.00024537300000000001,"stat_sum":{"num_bytes":48653353,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":36,"num_read":16743,"num_read_kb":125241,"num_write":4135,"num_write_kb":4319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":1065,"num_flush":178,"num_flush_kb":9852,"num_evict":474,"num_evict_kb":31566,"num_promote":521,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"162'2456","reported_seq":3354,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:11.588737+0000","last_change":"2023-09-29T21:32:11.588700+0000","last_active":"2023-09-29T21:32:11.588737+0000","last_peered":"2023-09-29T21:32:11.588737+0000","last_clean":"2023-09-29T21:32:11.588737+0000","last_became_active":"2023-09-29T18:25:04.586670+0000","last_became_peered":"2023-09-29T18:25:04.586670+0000","last_unstale":"2023-09-29T21:32:11.588737+0000","last_undegraded":"2023-09-29T21:32:11.588737+0000","last_fullsized":"2023-09-29T21:32:11.588737+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'2456","last_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_deep_scrub":"162'2456","last_deep_scrub_stamp":"2023-09-29T21:32:11.588616+0000","last_clean_scrub_stamp":"2023-09-29T21:32:11.588616+0000","objects_scrubbed":72,"log_size":2456,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:19:53.325596+0000","scrub_duration":0.24411396499999999,"objects_trimmed":0,"snaptrim_duration":0.00026188699999999998,"stat_sum":{"num_bytes":259907619,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":594,"num_read_kb":7561,"num_write":1805,"num_write_kb":1074526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":223,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"160'4364","reported_seq":12886,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:50.443532+0000","last_change":"2023-09-29T21:48:50.443532+0000","last_active":"2023-09-29T21:48:50.443532+0000","last_peered":"2023-09-29T21:48:50.443532+0000","last_clean":"2023-09-29T21:48:50.443532+0000","last_became_active":"2023-09-29T18:25:11.715402+0000","last_became_peered":"2023-09-29T18:25:11.715402+0000","last_unstale":"2023-09-29T21:48:50.443532+0000","last_undegraded":"2023-09-29T21:48:50.443532+0000","last_fullsized":"2023-09-29T21:48:50.443532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4364,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060073799999999997,"stat_sum":{"num_bytes":31827336,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":38,"num_read":16962,"num_read_kb":98002,"num_write":2272,"num_write_kb":1601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1416,"num_flush":223,"num_flush_kb":24092,"num_evict":496,"num_evict_kb":41546,"num_promote":546,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:49.432280+0000","last_change":"2023-09-29T21:48:49.432280+0000","last_active":"2023-09-29T21:48:49.432280+0000","last_peered":"2023-09-29T21:48:49.432280+0000","last_clean":"2023-09-29T21:48:49.432280+0000","last_became_active":"2023-09-29T18:24:59.529755+0000","last_became_peered":"2023-09-29T18:24:59.529755+0000","last_unstale":"2023-09-29T21:48:49.432280+0000","last_undegraded":"2023-09-29T21:48:49.432280+0000","last_fullsized":"2023-09-29T21:48:49.432280+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:24:58.520257+0000","last_clean_scrub_stamp":"2023-09-29T18:24:58.520257+0000","objects_scrubbed":0,"log_size":97,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"159'2445","reported_seq":3624,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:49:02.422240+0000","last_change":"2023-09-29T21:49:02.422240+0000","last_active":"2023-09-29T21:49:02.422240+0000","last_peered":"2023-09-29T21:49:02.422240+0000","last_clean":"2023-09-29T21:49:02.422240+0000","last_became_active":"2023-09-29T18:25:04.586658+0000","last_became_peered":"2023-09-29T18:25:04.586658+0000","last_unstale":"2023-09-29T21:49:02.422240+0000","last_undegraded":"2023-09-29T21:49:02.422240+0000","last_fullsized":"2023-09-29T21:49:02.422240+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2445,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001357446,"stat_sum":{"num_bytes":186970345,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":730,"num_read_kb":16921,"num_write":1884,"num_write_kb":1135789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"159'5112","reported_seq":10962,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:42.417443+0000","last_change":"2023-09-29T21:48:42.417443+0000","last_active":"2023-09-29T21:48:42.417443+0000","last_peered":"2023-09-29T21:48:42.417443+0000","last_clean":"2023-09-29T21:48:42.417443+0000","last_became_active":"2023-09-29T18:25:11.715338+0000","last_became_peered":"2023-09-29T18:25:11.715338+0000","last_unstale":"2023-09-29T21:48:42.417443+0000","last_undegraded":"2023-09-29T21:48:42.417443+0000","last_fullsized":"2023-09-29T21:48:42.417443+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":5112,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045015799999999998,"stat_sum":{"num_bytes":40416820,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":35,"num_read":12797,"num_read_kb":90880,"num_write":3364,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":6,"num_bytes_hit_set_archive":1482,"num_flush":191,"num_flush_kb":18832,"num_evict":504,"num_evict_kb":66860,"num_promote":551,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"162'1971","reported_seq":2861,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:32:09.485513+0000","last_change":"2023-09-29T21:32:09.485478+0000","last_active":"2023-09-29T21:32:09.485513+0000","last_peered":"2023-09-29T21:32:09.485513+0000","last_clean":"2023-09-29T21:32:09.485513+0000","last_became_active":"2023-09-29T18:25:04.584994+0000","last_became_peered":"2023-09-29T18:25:04.584994+0000","last_unstale":"2023-09-29T21:32:09.485513+0000","last_undegraded":"2023-09-29T21:32:09.485513+0000","last_fullsized":"2023-09-29T21:32:09.485513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1971","last_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_deep_scrub":"162'1971","last_deep_scrub_stamp":"2023-09-29T21:32:09.485385+0000","last_clean_scrub_stamp":"2023-09-29T21:32:09.485385+0000","objects_scrubbed":54,"log_size":1971,"ondisk_log_size":1971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:32:47.838060+0000","scrub_duration":0.18053175099999999,"objects_trimmed":0,"snaptrim_duration":0.00026822899999999998,"stat_sum":{"num_bytes":195391561,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":536,"num_read_kb":7690,"num_write":1794,"num_write_kb":1069671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'4519","reported_seq":12046,"reported_epoch":162,"state":"active+clean+scrubbing+deep","last_fresh":"2023-09-29T21:45:16.840893+0000","last_change":"2023-09-29T21:34:35.520997+0000","last_active":"2023-09-29T21:45:16.840893+0000","last_peered":"2023-09-29T21:45:16.840893+0000","last_clean":"2023-09-29T21:45:16.840893+0000","last_became_active":"2023-09-29T18:25:11.715369+0000","last_became_peered":"2023-09-29T18:25:11.715369+0000","last_unstale":"2023-09-29T21:45:16.840893+0000","last_undegraded":"2023-09-29T21:45:16.840893+0000","last_fullsized":"2023-09-29T21:45:16.840893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:10.583455+0000","last_clean_scrub_stamp":"2023-09-29T18:25:10.583455+0000","objects_scrubbed":0,"log_size":4519,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 766s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029783790000000002,"stat_sum":{"num_bytes":31929781,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":15715,"num_read_kb":84330,"num_write":3367,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1363,"num_flush":210,"num_flush_kb":5487,"num_evict":456,"num_evict_kb":42273,"num_promote":502,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'1765","reported_seq":2857,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:34:34.710494+0000","last_change":"2023-09-29T21:34:34.710471+0000","last_active":"2023-09-29T21:34:34.710494+0000","last_peered":"2023-09-29T21:34:34.710494+0000","last_clean":"2023-09-29T21:34:34.710494+0000","last_became_active":"2023-09-29T18:25:04.585685+0000","last_became_peered":"2023-09-29T18:25:04.585685+0000","last_unstale":"2023-09-29T21:34:34.710494+0000","last_undegraded":"2023-09-29T21:34:34.710494+0000","last_fullsized":"2023-09-29T21:34:34.710494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'1765","last_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_deep_scrub":"162'1765","last_deep_scrub_stamp":"2023-09-29T21:34:34.710383+0000","last_clean_scrub_stamp":"2023-09-29T21:34:34.710383+0000","objects_scrubbed":46,"log_size":1765,"ondisk_log_size":1765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:07:50.246535+0000","scrub_duration":0.14463637200000001,"objects_trimmed":0,"snaptrim_duration":0.00024837399999999999,"stat_sum":{"num_bytes":141160667,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":903,"num_read_kb":7637,"num_write":1590,"num_write_kb":1202293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":332,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"159'2089","reported_seq":3317,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-09-29T21:48:57.569189+0000","last_change":"2023-09-29T21:48:57.569189+0000","last_active":"2023-09-29T21:48:57.569189+0000","last_peered":"2023-09-29T21:48:57.569189+0000","last_clean":"2023-09-29T21:48:57.569189+0000","last_became_active":"2023-09-29T18:25:04.585681+0000","last_became_peered":"2023-09-29T18:25:04.585681+0000","last_unstale":"2023-09-29T21:48:57.569189+0000","last_undegraded":"2023-09-29T21:48:57.569189+0000","last_fullsized":"2023-09-29T21:48:57.569189+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T18:25:03.560821+0000","last_clean_scrub_stamp":"2023-09-29T18:25:03.560821+0000","objects_scrubbed":0,"log_size":2089,"ondisk_log_size":2089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055019400000000005,"stat_sum":{"num_bytes":289808522,"num_objects":80,"num_object_clones":1,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":1,"num_read":890,"num_read_kb":25574,"num_write":1933,"num_write_kb":1114911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":152827290,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":145,"num_read":62217,"num_read_kb":398453,"num_write":13138,"num_write_kb":10770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":19,"num_bytes_hit_set_archive":5326,"num_flush":802,"num_flush_kb":58263,"num_evict":1930,"num_evict_kb":182245,"num_promote":2120,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270319616,"data_stored":270150452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18375,"ondisk_log_size":18375,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":1721187941,"num_objects":487,"num_object_clones":2,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":2,"num_read":5931,"num_read_kb":92950,"num_write":14494,"num_write_kb":8889754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852,"num_omap_keys":85,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331472896,"data_stored":331291850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16730,"ondisk_log_size":16730,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740826,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246480,"kb_used_data":179464,"kb_used_omap":0,"kb_used_meta":67008,"kb_avail":94125360,"statfs":{"total":96636764160,"available":96384368640,"internally_reserved":0,"allocated":183771136,"data_stored":183006226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68616192},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.215,"15min":0.221},"min":{"1min":0.16900000000000001,"5min":0.14899999999999999,"15min":0.14899999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.0600000000000001},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24199999999999999,"15min":0.24099999999999999},"min":{"1min":0.21299999999999999,"5min":0.17899999999999999,"15min":0.153},"max":{"1min":0.437,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.20399999999999999}]},{"osd":1,"last update":"Fri Sep 29 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.246,"15min":0.24399999999999999},"min":{"1min":0.19700000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.74099999999999999},"last":0.224},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.26900000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.188},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.246}]}]},{"osd":0,"up_from":8,"seq":34359740824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250376,"kb_used_data":186432,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":94121464,"statfs":{"total":96636764160,"available":96380379136,"internally_reserved":0,"allocated":190906368,"data_stored":190154289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.26400000000000001,"5min":0.24299999999999999,"15min":0.223},"min":{"1min":0.14999999999999999,"5min":0.14999999999999999,"15min":0.14399999999999999},"max":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.27000000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.186,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Sep 29 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23400000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.157},"max":{"1min":0.41899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.222},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.25800000000000001},"min":{"1min":0.187,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.504},"last":0.23899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740822,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292212,"kb_used_data":226924,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":94079628,"statfs":{"total":96636764160,"available":96337539072,"internally_reserved":0,"allocated":232370176,"data_stored":231573860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 29 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.21199999999999999,"15min":0.21099999999999999},"min":{"1min":0.16200000000000001,"5min":0.13300000000000001,"15min":0.13300000000000001},"max":{"1min":0.35999999999999999,"5min":0.41599999999999998,"15min":0.435},"last":0.218},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.20000000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.39800000000000002,"5min":0.45200000000000001,"15min":0.47399999999999998},"last":0.26000000000000001}]},{"osd":2,"last update":"Fri Sep 29 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.22700000000000001},"min":{"1min":0.17499999999999999,"5min":0.16600000000000001,"15min":0.155},"max":{"1min":0.41999999999999998,"5min":0.436,"15min":0.44400000000000001},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.27100000000000002,"5min":0.28100000000000003,"15min":0.27600000000000002},"min":{"1min":0.20599999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.41699999999999998,"5min":0.47599999999999998,"15min":0.498},"last":0.26900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118480896,"data_stored":118418704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165736448,"data_stored":165645925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47255552,"data_stored":47227221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70475776,"data_stored":70441438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64684032,"data_stored":64633788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135159808,"data_stored":135075226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T21:49:04.403 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.404 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.404 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-29T18:24:58.520257+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=24, tm_sec=58, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.404 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.404 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.404 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-09-29T18:25:10.583455+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=10, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.405 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-29T18:25:03.560821+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=18, tm_min=25, tm_sec=3, tm_wday=4, tm_yday=272, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=29, tm_hour=21, tm_min=28, tm_sec=5, tm_wday=4, tm_yday=272, tm_isdst=0) 2023-09-29T21:49:04.405 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-09-29T21:49:04.447 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-29T21:49:04.448 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-29T21:49:04.448 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-29T21:49:04.448 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T21:49:04.462 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-09-29T21:49:04.459+0000 7f1738067700 -1 osd.1 162 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10617.0:1086 3.2 3:5c03df64:::rbd_data.29767bf0fbb3.0000000000000092:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3796992~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159) 2023-09-29T21:49:04.506 INFO:tasks.ceph.osd.0:Stopped 2023-09-29T21:49:04.507 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-29T21:49:04.507 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T21:49:04.593 INFO:tasks.ceph.osd.1:Stopped 2023-09-29T21:49:04.594 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-29T21:49:04.594 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T21:49:04.647 INFO:tasks.ceph.osd.2:Stopped 2023-09-29T21:49:04.648 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-29T21:49:04.648 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-29T21:49:04.648 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T21:49:04.673 INFO:tasks.ceph.mgr.x:Stopped 2023-09-29T21:49:04.673 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-29T21:49:04.674 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-29T21:49:04.674 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T21:49:04.731 INFO:tasks.ceph.mon.a:Stopped 2023-09-29T21:49:04.732 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-29T21:49:04.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | head -n 1 2023-09-29T21:49:04.796 INFO:teuthology.orchestra.run.smithi143.stdout:1696012238.4858449 osd.0 (osd.0) 3 : cluster [WRN] 5 slow requests (by type [ 'reached pg' : 5 ] most affected pool [ 'cache' : 5 ]) 2023-09-29T21:49:04.799 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-09-29T21:49:04.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T21:49:04.799 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-29T21:49:05.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T21:49:05.133 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-29T21:49:05.227 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T21:49:05.228 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-29T21:49:05.299 INFO:tasks.ceph:Archiving mon data... 2023-09-29T21:49:05.300 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834/data/mon.a.tgz 2023-09-29T21:49:05.301 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-09-29T21:49:05.315 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.KCuwpsKKvf 2023-09-29T21:49:05.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.KCuwpsKKvf 2023-09-29T21:49:05.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.KCuwpsKKvf 2023-09-29T21:49:05.540 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.KCuwpsKKvf is 424KB 2023-09-29T21:49:05.637 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.KCuwpsKKvf 2023-09-29T21:49:05.651 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-29T21:49:05.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-29T21:49:05.714 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-29T21:49:05.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-29T21:49:05.779 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-29T21:49:05.785 INFO:tasks.ceph:Archiving crash dumps... 2023-09-29T21:49:05.787 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834/remote/smithi143/crash 2023-09-29T21:49:05.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-29T21:49:05.849 INFO:tasks.ceph:Compressing logs... 2023-09-29T21:49:05.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T21:51:17.721 INFO:tasks.ceph:Archiving logs... 2023-09-29T21:51:17.722 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834/remote/smithi143/log 2023-09-29T21:51:17.723 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-29T21:51:29.060 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-09-29T21:51:29.062 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-29T21:51:29.077 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-09-29T21:51:29.079 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-29T21:51:29.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-29T21:51:29.134 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-29T21:51:29.143 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-29T21:51:29.143 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T21:51:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:bash: ntpq: command not found 2023-09-29T21:51:29.169 INFO:teuthology.orchestra.run.smithi143.stdout:210 Number of sources = 4 2023-09-29T21:51:29.169 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-29T21:51:29.169 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2023-09-29T21:51:29.169 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 -158us[ -163us] +/- 67ms 2023-09-29T21:51:29.169 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -372us[ -378us] +/- 54ms 2023-09-29T21:51:29.170 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +792us[ +792us] +/- 86ms 2023-09-29T21:51:29.170 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-29T21:51:29.170 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-29T21:51:29.184 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-29T21:51:29.185 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-29T21:51:29.198 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-29T21:51:29.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-29T21:51:29.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T21:51:29.334 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-29T21:51:29.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-29T21:51:29.359 INFO:teuthology.task.internal:Duration was 13241.215133 seconds 2023-09-29T21:51:29.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-29T21:51:29.375 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-29T21:51:29.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-29T21:51:29.419 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-29T21:51:29.705 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-29T21:51:29.706 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T21:51:29.706 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-29T21:51:29.725 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-29T21:51:29.725 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T21:51:29.805 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-29T21:51:29.805 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-29T21:51:29.920 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-29T21:51:29.931 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-29T21:51:29.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-29T21:51:29.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-29T21:51:29.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-29T21:51:30.019 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2023-09-29T21:51:30.032 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-29T21:51:30.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T21:51:30.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-29T21:51:30.096 INFO:teuthology.task.internal:Transferring archived files... 2023-09-29T21:51:30.097 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834/remote/smithi143 2023-09-29T21:51:30.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-29T21:51:30.163 INFO:teuthology.task.internal:Removing archive directory... 2023-09-29T21:51:30.164 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-29T21:51:30.211 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-29T21:51:30.222 INFO:teuthology.task.internal:Not uploading archives. 2023-09-29T21:51:30.222 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-29T21:51:30.233 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-29T21:51:30.234 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-29T21:51:30.265 INFO:teuthology.orchestra.run.smithi143.stdout: 263007 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 29 21:51 /home/ubuntu/cephtest 2023-09-29T21:51:30.266 INFO:teuthology.orchestra.run.smithi143.stdout: 658804 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 29 18:25 /home/ubuntu/cephtest/mnt.0 2023-09-29T21:51:30.266 INFO:teuthology.orchestra.run.smithi143.stdout: 658805 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 29 18:26 /home/ubuntu/cephtest/mnt.0/client.0 2023-09-29T21:51:30.266 INFO:teuthology.orchestra.run.smithi143.stdout: 658845 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 29 18:26 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-09-29T21:51:30.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T21:51:30.267 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-29T21:51:30.268 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_2c186684db7f9ea0cdb098fc86b350a0fdd26eba/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-09-29T21:51:30.279 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-29T21:51:30.291 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-29T21:51:30.302 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-09-29T21:51:30.302 INFO:teuthology.nuke:Checking targets against current locks 2023-09-29T21:51:30.331 DEBUG:teuthology.nuke:shortname: smithi143 2023-09-29T21:51:30.331 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T21:51:30.353 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_23:00:03-rbd-wip-yuri3-testing-2023-09-27-0919-quincy-distro-default-smithi/7405834', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 18:03:50.939568', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1dF7bKZeldZ/9Dl+g0zDsW2d5rAyu7pDHG6+AZnv2ETsbs1GwvhTB0ctyT6q5rNSs11+G17pq8WTLQX3HB918='} 2023-09-29T21:51:30.403 INFO:teuthology.orchestra.console.smithi143:Power off 2023-09-29T21:51:30.403 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T21:51:30.427 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2023-09-29T21:51:30.427 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T21:51:30.450 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T21:51:34.451 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T21:51:34.561 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T21:51:38.566 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T21:51:38.675 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T21:51:42.678 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T21:51:42.790 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2023-09-29T21:51:42.891 INFO:teuthology.orchestra.console.smithi143:Power off completed 2023-09-29T21:51:42.996 INFO:teuthology.run:Summary data: description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} duration: 13241.215132713318 failure_reason: 'Command failed (workunit test rbd/cli_generic.sh) on smithi143 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c186684db7f9ea0cdb098fc86b350a0fdd26eba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d6ea18ea60704a38bff424802b4f69d9 status: fail success: false 2023-09-29T21:51:42.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T21:51:43.078 INFO:teuthology.run:FAIL